Monday, October 5, 2015

Bayesian Analysis (4.5 out of 5 stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the  articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in October 2015 regarding Bayesian Analysis as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Bayesian Analysis is an statistical analytic method based on a mathematical theorem used to determine the probability of an event. The core of Bayesian Analysis is the use of updated knowledge as the analyst learns more information. As evidence is gathered, the more nuanced the estimate will get. As the process goes along, a higher accuracy of a probabilistic prediction will be produced.

  • Method gives a quantitative result to analysis.
  • By applying Bayes an analyst evaluates/weighs the new evidence and finds a revised probability by considering the previous one(s).
  • Method is a very structured technique that is easily replicated.
  • Method can provide a numerical value for words of estimative probability.
  • Method prevents vividness and recentness biases.

  • Can be manipulated by cognitive biases because it relies on subjective thinking.
  • Since this method requires the analysts to convert living events into the numbers, it may require some expertise/background to some extent in order to get meaningful results. However, this may not be called as the weakness, rather it could be a challenge for analysts.  
  • The math can scare people away from learning this technique.
  • Decision-makers at a tactical level may not understand this technique.
  • There is a time factor which can limit the practicality of using Bayes in some situations.
  • It is very hard to determine the importance of a piece of evidence before something happens.

While there are many different applications of Bayes, the basic formula is A=BxC or “the probability of an underlying cause (hypothesis) equals its previous probability multiplied by the probability of the observed event was caused by that hypothesis.” Since the exact probabilities are impossible to evaluate, estimated probabilities can be generated by using historical evidence or expert opinion to answer those complex questions.

The Bayesian formula is as follows:

Personal Application of Technique:
This methodology was applied through a scenario similar to the board game “Battleship.” In the scenario, which can be found in the “Bayesian Battleship” Powerpoint below, the class was an analyst for the Air Force in direct communication with a fighter pilot who is approaching a naval battle what has been divided into two sectors, the Lower-Right Quadrant (LRQ, 25 total squares, 25 empty), and the Rest of Board (RoB, 75 total squares, 51 empty). The analyst’s mission was to destroy the enemy submarine; however, we do not know where the enemy submarine is.

The problem was broken up into four parts representing added information to take into account. The first part is determining the probability that the enemy submarine is in the LRQ based on information that there is a 50-50 chance his sub is in either. The second part builds on the first, adding that the enemy submarine is about to fire on one of the allied ships, but the enemy submarine must be in a certain firing position (perpendicular to and centered on the enemy ship) which decreases the overall available locations of the enemy submarine.

The third part introduces the presence of an allied stealth submarine on the board, but we do not know where the allied submarine is, and it does not have any communications capabilities. Based on a recent sighting, there is a 90% chance that the allied submarine is in the LRQ. The problem is to find what is the probability that the allied submarine in the LRQ.

Finally, in the fourth part the pilot has an emergency where his missiles lock up and his aircraft is going down. Therefore, we must determine to which sector the pilot should set his autopilot, taking into account the main goal of destroying the enemy submarine and not hitting the allied submarine. The greatest likelihood is that the aircraft will hit the water and not hit either of the ships. The results are that in the LRQ, the aircraft has probabilities of 9.4% to hit the enemy submarine, 13% to hit the allied submarine, and 77.6% to hit the water. In the RoB sector, the aircraft has probabilities of 8.7% to hit the enemy submarine, 0.34% to hit the allied submarine, and 91% to hit the water. Based on these probabilities, the pilot should set his autopilot to the RoB sector because there is a significantly lower probability of hitting the allied submarine while only a slightly lower probability of hitting the enemy submarine. In either sector, the greatest probability is that the aircraft will hit the water.

For additional information:
Bayes Theorem and Intelligence

Saturday, October 3, 2015

Bayesian Analysis for Intelligence: Some Focus on Middle East

By: Nicholas Schweitzer


    Nicholas Schweitzer introduces this report for the CIA by briefly discussing one problem intelligence analysts face today: the 'information explosion'; intelligence analysts struggle to filter through the copious amount of information with their limited resources. Another problem Schweitzer acknowledges is the difficult questions analysts have to answer in an increasingly complex world. One way to ease these problems is through the application of analytic methods.
    Schweitzer introduces the Bayesian analysis as a simple mathematical equation: A = B x C. He explains it further as "the probability of an underlying cause (hypothesis) equals its previous probability multiplied by the probability of the observed event was caused by that hypothesis". Schweitzer describes how analysts use Bayes to predict complex political outcomes. Analysts use estimated probabilities since the exact probability is impossible to evaluate. To generate these probabilities, analysts can use historical or expert opinion. Schweitzer even says analysts can incorporate the Delphi method, which uses expert opinions, alongside Bayes. The approximated probabilities provide a "starting point" of analysis    


  • Provides a wide range of expertise instead of depending on a single analyst
  • More information can be extracted from data
  • The process is reproducible
  • The end result can be displayed on a numerical scale
  • Analysts are forced to look at other perspectives


  • Can only be applied to certain questions (i.e. war or no war?) 
  • Must be expressed as a specific set of hypothetical outcomes
  • Must follow mathematical principles 
  • Needs to have a large amount of data which increases the probability of unreliable or even negative information
  • Can be manipulated by cognitive bias
  • Cannot be applied to crisis situations due to the need for time

     The author uses great examples, especially to stress the complexity of intelligence questions and defining the kinds of questions that work with Bayes. I also liked how Schweitzer discussed its applicability with the Delphi method because it is unlikely an analyst will only use one technique to come to a final conclusion on a difficult issue. However, there was not much explanation into the process of setting up Bayesian analysis (especially regarding the examples of the Middle East), and interpreting the results. I think this report does provide readers with a basic grasp on how Bayes works, but would not be enough of an explanation for analysts to use as a how-to guide. 

Friday, October 2, 2015

Using Bayes’ theorem in behavioural crime linking of serial homicide

By: Benny Salo, Jukka Siren, Jukka Corander, Angelo Zappala, Dario Bosco, Andreas Mokros and Pekka Santtila


This article discusses the importance of crime linking, which is basically drawing the conclusion that the same offender is responsible for more than one crime, to law enforcement and describes a study in which Bayes' theorem was used in order to do this with a high degree of accuracy. Crime linking has the potential to benefit law enforcement greatly because if multiple crimes can be linked together in a reliable way, then the information from the investigation of the separate crimes can be combined which allows for further conclusions, a greater amount of investigation methods, and the efforts to solve each separate crime can be joined together. Because crime linking can be an invaluable asset when presenting in court as well as guiding analysis, it is necessary that the methods used to link the crimes are effective and that their theoretical assumptions can hold true. Since Bayes' theorem is considered a "revolutionary" method in other disciplines, the authors wanted to see if it would be viable for crime linking as well when it is applied to the behavior of offenders which is why this study was conducted.

This article states that a strength of using Bayesian reasoning in this discipline is that "Bayesian reasoning provides a coherent framework for handling uncertainty, within which the individual behaviours can be weighed against each other to reach a conditional probability of any particular crimes being linked" (Salol, 2012). Bayesian reasoning has been used before in other studies with empirical Bayesian approaches which did not yield significant results, but this study used a fully Bayesian approach which hadn't been done before.

 In this study, 116 homicides belonging to 19 separate series of homicides were analyzed in regards to number of victims, time period in which the homicides occurred, age of the offender, etc. Basically all of the details of the crimes were analyzed using a Bayesian model that assigned different probabilities on observing specific behaviors based on which series that the crime belonged to. The researchers used a leave-one-out cross-validation scenario (LOOCV) in order to test the model. When the results of the scenario were analyzed, it was found that the Bayesian model correctly classified 83.6% of the cases. All in all, the researchers seemed to be very impressed with the accuracy of the Bayesian model and feel that it is a promising method to use for crime linkage as it can distinguish even minor behavior variations and is useful in the beginning of the series which is very important.


I thought this article did a very good job of explaining how useful and accurate the Bayesian model can be when it comes to linking crimes, but it did not go into very much detail about how exactly the researchers went about doing this. It did not actually show the model or explain how it worked which I think would have been beneficial. Other than that, however, the article did a pretty good job of critiquing itself as it mentioned all the limitations of the study in the discussion section. Overall, I found this article to be extremely interesting because it delves more into law enforcement intelligence and crime analysis perspectives which I appreciated.


Bayes Theorem and Intelligence

Beliefs are based on probabilistic information in Bayes Theorem.  This method is used to measure incomplete knowledge and uncertainty. After observing new conditions, our initial beliefs are updated. At a glance, one can see that Bayes Rule identifies our initial beliefs as having a high margin of error. Analytic confidence or rigor increases as we observe more conditional events. According to the author, Bayes Theorem is “highly subjective and somewhat controversial compared to more objective probability theories in statistics.”

·         The author’s main argument is that “Intelligence analysis uses Bayes Theorem and is very subjective.” Below is a list of some of his chief arguments:

  • The prior belief is updated to a posterior belief after the observation of conditional events.
  • This is a variant of the Bayes Rule formula, where P is probability, C is the conditional event, and O is the observation, and ¬ is not.    
  •  p(C|O) = p(O|C)p(C)/p(O|C)p(C) + p(O|¬C)p(¬C)
  •  In English, the posterior belief is equal to the prior belief divided by the marginal probability.
  • Bayesian probability produces interesting results because it accounts for uncertainties created by False Positives and False Negatives.
Intelligence uses predictive analysis to predict a range of events in the face of extreme uncertainty. Yes, it is possible. It predicts the probability of events, but it cannot predict which events will occur.

In the case of law enforcement, suspects are carefully vetted several times and on occasion this broadens their field of suspects by the accumulation of evidence while increasing their certainty.

This is why Police carefully investigate suspects many times, and occasionally widen their field of suspects if they believe their initial investigation led them in the wrong direction. The following is an example breakdown a law enforcement application of Bayes:

There are 10,000 civilians. 1% of whom are insurgents pretending to be civilians. Police can investigate individuals and determine if they are an insurgent or civilian with 95% certainty.

Prior Probability is this: 0.01 (10,000) and 0.99(10,000). So
Group 1: 100 insurgents
Group 2: 9,900 Civilians
The Police investigate the entire population. This produces four groups:
Group 1: Insurgents – Positive test (0.95)
Group 2: Insurgents – False Negative test (0.05)
Group 3: Civilians – False Positive test (0.05)
Group 4: Civilians – Negative test (0.95)

How certain are the police that the men they captured are actually insurgents? The answer is 16%.
(0.95 x 0.01)/ (0.95 x 0.01) + (0.05 x 0.99) =
0.0095/0.0590 = 0.161

According to the author, the result is very counterintuitive due to the high level of uncertainty created by false negatives and false positives. For example:

“If police investigate the entire population, up to 6% of the population will test positive for being insurgents. But we know only 1% of them can be insurgents and the others are innocent. We also know some insurgents may have escaped detection. Thus the 16% certainty.”

Critique: Overall, I agree with the author’s position on Bayes Theorem and intelligence. He also seeks to address the wrong perception that intelligence analyst predict future events. The application of Bayes theory for detecting a black swan, for instance, is near impossible with the application of Bayes. This is due to Bayes reliance on scenarios which have already happen. Despite the fact Bayes will not predict a black swan, Bayes remains useful for forecasting.

Source Link:

What is Bayesian Analysis?

"What is Bayesian Analysis?"
Contributed by: Kate Cowles, Rob Kass, & Tony O'Hagan
International Society for Bayesian Analysis


This article is a brief overview of the history of Bayesian Analysis and a basic breakdown of the mathematics behind the method. The article says Bayes has been around since the late 1700s, however it's popularity and practicality of practice were severely strained. Modern Bayesian analysis as we know today was developed in the second half of the 20th century by Jimmy Savage and Dennis Lindley. Still, fully developing the method was difficult until the breakthrough of the computer age in the 1980s and 1990s. Finally, statisticians had the computational ability to handle the mass data and complex equations needed to do Bayesian analysis.

The breakdown of Bayesian Analysis is at a basic level is an estimator that estimates the probability of a hypothesis coming true as evidence is gathered. The method starts out by using the "prior distribution". As more data becomes available, also known as the "y", it contains the parameters and it is expressed in a "likelihood" which is set in proportion to the observed data. This new data is combined with the prior equation to formulate a "posterior distribution". Long story short, this is a method of probability theory, and the evidence and corresponding likelihood is all calculated. The authors argue the major benefit of Bayes Analysis is the analysis itself is very objective and can be applied to a wide variety of scenarios. The weakness the authors point out is that Bayes is still built on a prior distribution, which can be very subjective.


My major issue with the article is that it does not do a good job of demonstrating how Bayes is used in practicality. At a basic level I can see that it is used to predict outcomes and the probability of a hypothesis coming true, however a real-life example or even a logical but fictional scenario would be very helpful in showing the public how Bayes works. I would also say that Bayes if possible should have the mathematics broken down in a simpler level so the general public could maybe learn the basics and then possibly be encouraged to delve deeper. The overall complexity of the method can scare even people with some statistical background in my opinion. I enjoy the idea of using math to form an estimate, but the practicality of actually executing the method is difficult in my opinion.



Thursday, October 1, 2015

Bayesian Intelligence Analysis

Davide Barbieri


The author shows how Bayes’ method work for intelligence analysis firstly by defining the other probability measurements. The study’s objective is to give a general idea of Bayes’ method by showing the logical overlap and differentiated points with other mathematical measurement.

The author mentions about the simple probability approach which  was made famous by French mathematician Laplace: it says that the probability P of an event E is the number of favorable cases m divided by the total number of possible cases n:  P(E)=m/n
This classic definition can be applied when all the possible elementary outcomes of a random trial (or experiment) are known and each of them has the same chances of happening. For example, the probability that the roll of a die will give an even number is 3/6=1/2.
If things become less obvious then the conditional probability technique can be used which defined as: P (A | B) = P (A and B) / P (B) Chances that an event will occur given another event, that is the imposed condition: P(A|B), which can be read as the probability of A given B. For example, the probability of any given number in a fair die is 1/6. The joint probability that the roll of a die will give an even number n greater than 3 is P(AB), where A is the event “n is even” corresponding to the following outcomes: {2, 4, 6}, and B is the event “n>3” corresponding to {4, 5, 6}. Therefore, their intersection is (AB) = {4, 6}, and the corresponding joint probability is calculated as 2 favorable cases out of 6 possible: P(AB) =2/6=1/3. Since the probability of B is P(B)=3/6=1/2, then the conditional probability that n is even given B is P(A|B)=P(AB)/P(B)=(1/3)/(1/2)=2/3. In fact there are 3 numbers which are greater than 3 in a die: {4, 5, 6}, 2 of which are even: {4, 6}
Joint probability
Finally, the author comes to the Bayes’ rule. And, he suggests that “In science, and in medicine in particular, researchers want to know the probability of an event given some evidence.” This sentence actually forms the foundation of what Bayes’ method seeks for. Regarding the intelligence perspective, in a paper by Zlotnick (1970), Bayes’ rule is defined in the following way: R = PL where R is the estimate of the conditional probability of hypothesis H after revising the latest evidence E. R is equal to P, the prior estimate (which is given) times L, the likelihood ratio of event E in case hypothesis H is true:
The author mentions about some example studies that the analysts utilized Bayes method to have an estimate. The first one is: In August 1969 the CIA had to evaluate the hypothesis that the USSR would attack China within the following month, in order to destroy its alarming nuclear capabilities. Analysts were asked to make an estimate, that is to evaluate the probability of a war. A list of intelligence items (Evidence) was collected such as E1, E2 … En. Then the analysts evaluated from their past experience the likelihoods P(Ei|H) of E1, E2... En in case of a war. They were asked to revise their estimates every week as the new evidence were available. Eventually, Bayesian probabilities always fell below conventional probabilities, demonstrating a better predictive accuracy.
Conventional and Bayesian probabilities (Adapted from Fisk 1972)
Another example: There are two hypotheses. Israel was not going to attack Syria (H0, the null hypothesis) or that Israel was actually going to attack (H1, the alternative hypothesis). Being two complementary events, the sum of their probabilities had to be 1.
The prior probabilities of the two competing hypotheses were set to P(H0)=0.9 and P(H1)=0.1, with war being unlikely. An additional piece of information was then revealed: “Israeli finance minister Rabinowitz stated that the nation’s economic situation is one of war and scarcity, not one of peace and prosperity”. After hearing the minister’s statement on the radio, the two likelihoods of such an event were estimated by the analysts as P(E|H0)=0.8 (in case of no war) and as P(E|H1)=0.99 in case of war. Then they applied Bayes’ rule and revised the probability of an attack accordingly:
P (H0|E) = P(E|H0)P(H0) / P(E) = 0.8 x 0.9 / 0.819 = 0.88
P (H1|E) = P(E|H1)P(H1) / P(E) = 0.99 x 0.1 / 0.819 = 0.12
P(E) = P(H0)P(E|H0) + P(H1)P(E|H1) = (0.9 x 0.8) + (0.1 x 0.99)=0.72+0.099= 0.819
The perceived risk increased consistently according to the analysts’ view.
To sum up, we find the revised probability of the previously stated probability in case of additional available information by applying the rule.
The study asserts that the Bayes’ approach can give strategic warnings, can force analysts to quantify their estimates in numerical values, it may reduce cognitive bias. Since analysts are usually better at evaluating a single piece of evidence at a time rather than at drawing inferences from a large body of evidence, the method can enable them to focus thoroughly on one single evidence each time.
The technique is very useful in terms of taking into account many different evidences that seemingly aren’t related with each other, whereas the classical probability approach deals with mainly related events (e.g. rolling a dice: you always roll the same dice and already every dice has the same features). Moreover, we use WEPs while articulating our estimates. But, Bayes’ approach enables analysts to give more accurate estimates rather than giving a range. On the other side, the method itself requires some talent in statistics and learning and applying the method is tough. The technique still requires analysts’ perceptions and experiences at the inception and while forming the formula (as assessed in article, there is a given probability which is the evaluation of the analyst about the topic. And then, after applying the rule we find the revised probability of the previously stated one). Therefore, to some extent, we can say that if the confirmation bias is present while weighting the evidences, the results shall not reflect a %100 ( or almost) accurate estimate.

Monday, September 28, 2015

Speed Reading (4 out of 5 stars)

Note: This post represents the synthesis of the thoughts, procedures and experiences of others as represented in the  articles read in advance (see previous posts) and the discussion among the students and instructor during the Advanced Analytic Techniques class at Mercyhurst University in September 2015 regarding Speed Reading as an Analytic Technique specifically. This technique was evaluated based on its overall validity, simplicity, flexibility and its ability to effectively use unstructured data.

Speed reading is a modifier that allows one to read a book, journal article, novel or magazine at a faster rate, but, possibly, at the cost of reading comprehension. Likewise, the modifier needs to be practiced and used on a consistent basis in order to maintain the benefits of the technique, otherwise the skill deteriorates. While the skill is useful, more investigation needs to be done into which techniques and training that improve speed and comprehension, particularly if the success of these techniques vary between individuals. Along the same lines, more research is needed to specify which techniques are most beneficial to intelligence analysts.

  • With consistent practice, speed reading has the potential to increase efficiency.
  • Increased reading speed increases the amount of material an analyst can get through thereby increasing their overall efficiency.
  • Enables people to rapidly read through large amounts of material.
  • Training and practice allows analysts to comprehend as much or more information relative to average readers.

  • Requires training and consistent practice to read faster while comprehending more information.
  • Eliminates emotion from reading.
  • Prevents checking previously stated information via regression.
  • Potential to miss details and decrease comprehension without experience.
  • All techniques may not be applicable to all readers
  • Results may vary between mediums (computers and books).

While there are many different methods of speed reading, the class was exposed to the ones listed below:
  1. A popular method is to use a finger, pen, or computer mouse to follow along the words, thus forcing the individual to speed up to their pointer.
  2. Read paragraphs in chunks, identifying key words and phrases and thus identifying the gist of the paragraph.
  3. Remove subvocalization. Instead of reading along to the voice in one’s head, read to the pace of one’s finger or pen.
  4. Eliminate re-reading words, phrases, & sentences.
  5. Use your peripheral vision. Pretend there is a vertical line in the middle of the text and focus on that line while using one’s peripheral vision to see and comprehend the words on either side of the line.

Personal Application of Technique:
In order to apply this technique, the class was split into two groups and there were two rounds of testing. In round one, one group was exposed to a training video about speed reading techniques and one of the groups was not. Then, each person read a passage to test their words per minute (WPM) and took a comprehension test. With that data, efficient words per minute (EWPM) was calculated and the results of the two groups were compared.  

Testing and Practical Exercises
Using the links below, the analysts speed reading performance was measured. This tested both words per minute and reading comprehension. These results produced the effective words per minute (EWPM) scores. Once scores were calculated on the excel spreadsheet, the group was able to discuss the results.

After a discussion of the results, round two was conducted. In this round, every person in both groups was exposed to speed reading techniques. One technique at a time was exposed to the subjects and they were to apply that technique to a passage and answer three questions about it. After that, the data gathered was used to again calculate EWPM and the results were compared with those of round one.

For additional information:
We used the excel spreadsheet to visualize the results in order to show the effects of two-minute training video.