Saturday, November 11, 2017

The Problems with Monte Carlo Simulation

Summary and Critique by Keith Robinson Jr.

Summary

David Nawrocki, professor of Finance at Villanova University, examines the failings of Monte Carlo simulation in the financial realm. The author explains that Monte Carlo simulation is only useful in situations where data and analytic models are unavailable; we possess some knowledge about the population, however, sampling data is unavailable. Rubinstein (1981) explains Monte Carlo simulation is appropriate when:

  • It is impossible or too expensive to obtain data
  • The observed system is too complex
  • The analytical solution is difficult to obtain
  • It is impossible to validate the mathematical experiment
Further exploring the extant literature at the time, Nawrocki found no articles in support of using Monte Carlo simulation with financial return markets. Additional literature suggested that the while the methodology educates people on uncertainty and risk, it does not reduce uncertainty, instead increasing it because it is derived from assumptions which can lead to incorrect decisions. Implementation is not easy. Nawrocki provides a number of cases in which Monte Carlo simulation is unnecessary or fails. Because Monte Carlo simulation assumes all distributions are normal and correlations are zero, it does not accurately capture the interrelationships between multiple variables contained in historical data, therefore, it does not depict real-world complexities.  


He suggests that Monte Carlo simulation lacks an adequate benefit/cost ratio and provides no demonstrably better answers than other analytic techniques. The researcher argues that if a number is derived from unrealistic assumptions, then they possess no real value. Regarding policy implications, Nawrocki explains that the best policy adapts to uncertain conditions, rather than relying on the most likely course of action produced by the simulation. So, while useful for cases where data and analytic models are unavailable, Monte Carlo simulation requires more work and does not necessarily produce better answers than other analytic techniques.

Critique:

When analyzing Monte Carlo simulation's for use in intelligence, it very well could be a powerful tool for assessing risk, but not necessarily for reducing uncertainty.  Monte Carlo simulations can provide the most likely outcomes, but that does not necessarily reflect real-world scenario. Assuming an adversary/competitor will proceed down a specific path is comparable to mirror imaging bias. While an outcome may be the most likely, one cannot assume all actors are rational actors.

Source: Nawroki, D. (2001). The Problems with Monte Carlo Simulation. Journal Of Financial Planning, 14(11), 92-106.

Citations:

1. R. Y. Rubinstein, R. Y., (1981). Simulation and the Monte Carlo Method. New York: John Wiley and Sons.

3 comments:

  1. It appears that majority of the Monte Carlo Simulation (MCS) problems emanate from its application, not the process. As a result of the repetitive nature of the process, inappropriate application of MCS may produce results that are not close to real world scenarios, which can render MCS ineffective and ridden with problems.

    ReplyDelete
  2. It does seem that Monte Carlo simulations presume that all distributions are normal and correlations are zero, this in turn does not connect interrelationships among multiple variables contained in historical data, essentially not portrying real-world complexities.

    ReplyDelete
  3. The author mentions "if a number is derived from unrealistic assumptions, then they possess no real value." For this reason, I could see that this method still relies to a degree on subject matter expertise and therefore, is best used in conjunction with other techniques.

    ReplyDelete