I’ll place my cards on the table here. I love Monte Carlo sims. I think they are an incredibly handy method for exploring complex risk spaces. But I have a suspicion that there are rogues out there, who believe that you can use MC to quantify things you don’t know.

What do I mean?

Well, MC methods work like this. You have a system, which you understand. And its final state is dependent upon things that could take on a range of values which you don’t know. Watch it! There it is! You don’t know the values, which is different to not understanding the range of values.

MC models are incredibly sensitive to the question you ask of them, and you ask this question through the structure of the inputs. Using a vague input will give you a vague answer. If certain inputs are “more likely” than others, and you don’t correctly enforce this, your model will be skewed.

As an example, we often use Monte Carlo methods to understand Bayesian interpretations of statistics. Bayesian simply means the probability of an hypothesis, given some data. It is different to frequentist statistics, which answers the question regarding the probability of data, given the hypothesis. We use MC methods here, because the number of hypotheses are large, and their manifestations complex.

But what happens if you vaguely guess a parameter that affects your hypothesis? Say you are trying to understand failure rates, and one input is a common garden tap. If you let it pump up to 1000 gallons a minute in your model, what do you think that does to the interpretation? If you are modelling the overflow of the tank, then this model is ok, as the parameter will allow this to happen perfectly well. But if you want to model failure times, then the model is useless, as the tap is physically unable to produce such a flow.

Now I know no-one reading this would ever make such an assumption, but think carefully about some of the subtler ones in the model, and even how they might interact to twist the validity of the interpretation.

And to that particular practitioner who likes to use big words, and large numbers of repetitions to confuse policy makers with highly precise, yet inaccurate models…….I’ve got your number