How Climate Model Uncertainty Should Influence Climate Policy

“All models are wrong; some models are useful” –  George Box

“I don’t believe anything, but I have many suspicions” – Robert Anton Wilson

Climate models are our primary method for predicting the future state of the Earth, and so are a crucial influence on climate policy.  Politicians often demand firm evidence that climate change is real. Scientific evidence of climate change has been around for decades, however, skeptics still manage to blow any uncertainty in scientists’ models out of proportion. In the following post I will discuss a short article by J. Norman et al. in which the authors argue for action to mitigate climate regardless of whether we have perfect models.

In the article they pose a powerful question which helps us think critically about what factors come into our decisions about climate policy. This question can be phrased as: “what would the correct policy be if we had no reliable models?” Thinking carefully about this question allows them to dismantle the assumption that it is only worth acting to combat climate change if you believe in climate scientists predictions. This is important to consider since many climate skeptics arguments rely on pointing out the flaws in scientists models. Furthermore, this question represents an interesting limit case to think about since it encourages us to think about how we should behave given that there are uncertainties in our models.

Their argument is based on the so called precautionary principle which, from a risk management perspective, posits that if something is potentially harmful to the public, the burden of proof on the people who want to carry it out rests on establishing that it isn’t, not the other way round. In short: we shouldn’t dabble with things when we don’t know what the unintended consequences might be. This is especially true when there is even a small risk of a complete catastrophe. Ignore this principle at your own risk in everyday life, but when we alter things that might affect whole ecosystems or planets it is definitely worth being extremely cautious about the consequences of our actions, which may not be reversible.

Any predictive model will have uncertainty in its outcome. In addition to this uncertainty, we must also strive to remember that the model isn’t reality, no matter how hard we try, meaning that there will always be events which are out of the range of our predictive powers. The importance of events which are out of our predictive range was popularized by Nassim Nicholas Taleb in the 2007 book The Black Swan. “Black Swans” are extreme events that lie out of our predictive range (no matter how good our supercomputer is). In the management of risk, the impact of these events may also be referred to as Knightian Uncertainity, this is a risk that is essentially uncomputable.

Just as an unpredicted market crash will render forecasts produced by economists for the future obsolete, there may also be unexpected events which will have a dramatic effect on current climate predictions. For example, a huge volcanic eruption might occur or some new carbon capturing technology might conceivably allow us to remove a large amount of CO2 from the atmosphere in a matter of months. In fact, we even have examples of relatively sudden changes in atmospheric composition happening in the past, as is the case with the “Paleocene-Eocene Thermal Maximum”.  Taleb’s message in the Black Swan is not that we should try and predict these events, but that we should instead be aware that, good or bad, unknown unknowns are out there in the future, whether we like it or not.

Figure 1. [3] Forecasting Skill plotted against year for different forecasts. This demonstrates the improvement of our ability to predict weather patterns over relatively short time scales. We shouldn’t naturally assume that this transfers over to the quality of our climate predictions. However, these types of observations provide evidence of the plausibility of physical models in the prediction of future states of the climate system.
That being said, we shouldn’t be totally pessimistic about our ability to predict. Weather and climate models can and have been statistically tested to perform fairly well [2]. In addition, the accuracy of weather forecasts and our ability to “see” longer into the future has indeed improved over the years (see figure 1). Given this, we should be careful not to misinterpret Norman et al.’s article as saying “Policy makers should never use climate models because we simply can’t predict things.” The main point is summarized in the final two sentences of the article:

“The popular belief that uncertainty undermines the case for taking seriously the ’climate crisis’ that scientists tell us we face is the opposite of the truth. Properly understood, as driving the case for precaution, uncertainty radically underscores that case, and may even constitute it.”

There will be plenty more to come on the blog about this difficult situation! For now, the main thing to take away is that we need to keep an open mind about the different possible outcomes which might arise in the future, predicted or not.

In summary:

  • We shouldn’t blame scientists for failing to include the presence of certain rare events in their models. After all, with limited computing power there is only a certain amount one can include in a model of such a massive system.
  • However, we can blame scientists or policy makers if they attempt to implement policies which ignore the possibility of anything that the model doesn’t explicitly predict.
  • Uncertainty is reason to act, not a reason to not act.

“The ancients knew very well that the only way to understand events was to cause them.”  – Nassim Nicholas Taleb


References and Further Reading:

[1] Taleb, Nassim Nicholas. The black swan: The impact of the highly improbable. Random House, 2007. Highly recommended for anyone who wants to read more about the impact of rare events on the course of history. Taleb goes into detail about the reasons why we should be very careful when transferring predictive models inspired by those in the physical sciences into domains such as the social sciences and economics.

[2] For more on how good climate models and their predictions have been for us so far:

[3] Image Reference:

[4] Cover Image:


What on Earth is Mathematics of Planet Earth?

Until recently I had no problem explaining what I was studying; I was just an average maths student. I could reliably predict the reaction of a person informed of this fact: “How can you do that? I’ve always been hopeless at maths. And anyway, what are you going to end up doing with that degree”? Things got much more complicated when I started a PhD at Mathematics of Planet Earth. The first reaction is now usually the question used as a title to this article: what on earth is mathematics of planet earth?! After a brief explanation that I am learning how to use and develop maths for climate and weather predictions, I just get a reassuring statement: “I know that this whole climate change thing is very dangerous/rubbish” (choose the option that applies to you) and a question: “But why did you resign from doing proper maths?”

Actually, I am more involved in studying mathematics than ever. No science would exist without mathematics, in particular climatology or meteorology. Some people can predict the rain by feeling it in their bones; I can “predict” the rain more or less based on the fact that we are in UK. But do we really want to risk our life on someone’s body niggles? No, I do not exaggerate. Our life really can depend on it. Do not forget that a bad weather prediction not only can get you wet, but also farmers might not prepare for a drought (and the crops would get extremely expensive next year), local authorities might not decide to grit ice-covered roads (so you might get stuck in traffic or even have an accident) or a dangerous storm might hit citizens completely not ready for it. It is something worth looking at, is it not?

To get more reliable predictions about the state of atmosphere in the next couple of hours, days or even centuries, we need… mathematics. No, not that boring multiplication table, but nearly every field of very advanced mathematics. Let us take a look at a couple of examples.

Chaos Theory

You will have heard of the “butterfly effect” which allegedly can provoke a hurricane. This is all about the chaos. The intuitive definition, given by E. Lorenz, the creator of chaos theory, is [1] when the present determines the future, but the approximate present does not approximately determine the future. It means that if we were given infinitely precise initial conditions (i.e. full description of the state of the weather now), we could predict the weather at any time in the future. So why do meteorologists sometimes get it wrong? Because this is just wishful thinking. In reality we are not able to get perfect measures of the weather components, for example due to the limitations of measuring devices. Thus mathematicians need to choose the most important measurements with the available precision and try to get the best prediction they can. However, chaos theory states that, under some conditions, starting from almost the same state we can get completely different results. It complicates weather prediction so chaos theory is still something we need to study.

Numerical Analysis

There would be no weather forecast without very advanced computers we are using. Some of them are even supercomputers, such as the one used by Met Office. It costed a trifling £97 million. Why do governments invest such enormous sums into such equipment? Before we understand that, we have to see how the weather prediction works. As mentioned above, we cannot forecast it exactly. Hence mathematicians have to get rid of some parameters that seem to be less important (by the way, deciding which are those is far from obvious) and, using the ones that are left, build a model. This is a set of equations (sometimes thousands of them!) that describe the system. Do you remember solving systems of two equations at school? You might have struggled with it. So now imagine solving thousands of much more complicated ones. Yes, this is exactly why we need supercomputers; they make this job feasible. However, mathematicians still need to make sure that the result produced by a computer is sensible. They do it by carrying out a numerical analysis, checking the properties of the system.

I’ve mentioned only a tiny fraction of the whole range of mathematical tools used in the weather prediction. Next time when you listen to your favourite weather forecast, keep in mind that it would not make any sense without mathematics. And if you happen to have a child, encourage them to study maths. Just in case.


[1] Danforth, Christopher M. (April 2013). “Chaos in an Atmosphere Hanging on a Wall”. Mathematics of Planet Earth 2013. Retrieved 27 January 2016.

It’s OK When Weather Forecasts Are Wrong

Weather forecasts don’t have a great rep. Since I started studying the Mathematics of Planet Earth, I’ve lost count of the number of friends and family members that have asked me, “But weather forecasts aren’t any good are they?” Sure, weather forecasting isn’t an exact science. You only have to follow a forecast for a week in Autumn or Spring in the UK to notice that it rains when you were told it would be sunny and vice versa. Have a look at Figure 1, which shows the number of times the Met Office correctly forecasted rain throughout 2014 — they get it right about 70% of the time (this is very good — the Met Office claim to be second in the world for quality of their forecasts [1]). However, I don’t think this is a problem with the science. It’s a problem with how we interact with the forecasts.

Figure 1: The fraction of times rain was accurately forecast by rain symbols one day ahead of time by the Met Office throughout 2014. Source of data: a/pdf/5/6/MOSAC20_2015_Anne x_III_forecast_accuracy.pdf
Figure 1:
The fraction of times rain was accurately forecast by rain symbols one day ahead of time by the Met Office throughout 2014.
Source of data: a/pdf/5/6/MOSAC20_2015_Anne x_III_forecast_accuracy.pdf

In fact, weather forecasts should not be correct all the time. Ewen McCallum, a former Chief Forecaster at the Met Office says: “If we got it [the weather forecast] right every time, we’d be God.” Trying to predict the weather is a bit like trying to predict dice rolls — we just don’t have enough information to be able to do so. In order to have a perfect prediction we would need to know the exact air conditions — temperature, pressure, wind velocity and humidity — at every single point in the Earth’s atmosphere with perfect accuracy. This clearly isn’t possible: even if our measuring instruments were perfect, it’d be impossible to know these properties everywhere. Even the flap of a butterfly’s wings or a baby’s breath will cause these quantities to change.

Surely these effects are too small to make a difference though? It turns out this isn’t true. The Earth’s weather is known as a complex system — the tiniest changes to its state cause the system to evolve in a different way. This phenomenon is known as chaos and is observed in many types of system. It means that no matter how close we get to measuring or guessing the current conditions, at some point in the future we won’t be able to determine the weather.

One of the ways that forecasters try to account for this is by trying their simulations many times with slightly different conditions at the start, to account for the unknowns in their measurements of the current state of the atmosphere. This technique is known as ensemble forecasting. Rather than telling forecasters what will definitely happen at 3pm on 19th of April, they might see that in 70% of their simulations it rains over London at this time. This gives them a probability for it raining, rather than a definite answer to whether it will or won’t.

However, we as the public don’t like this kind of uncertainty. We simply want to know if it will or if it won’t rain, and it almost looks weak of a forecaster to avoid giving a definite prediction. Unfortunately giving a definite prediction is poor science as it does not represent truth about the weather. Given that the nature of the weather is unpredictable, demanding a definite forecast will inevitably lead to failure in the long term.

So the problem is not so much with the quality of the research at the Met Office — it lies with our expectation for a definite prediction rather than one that contains uncertainties (such as one saying there is 70% chance it will rain today). These uncertainties are an inherent part of a chaos theory, which we see in the weather as we can’t have perfect knowledge of the whole system at once.



Why bring maths into it?

To thrive on planet Earth, knowing what it’s going to throw at us is key. We need to know what crops can be grown where and what time of year to plant and harvest them. We need to know whether the mosquito that’s about to bite us is likely to be carrying malaria. We need to know how to manage flooding, and how much snow or wind our structures have to withstand.

So how do we know these things? Experience, first. The security that what will happen tomorrow will probably not be too much different from things that have happened before. But more and more, we are profiting from a delicate and precise understanding of the “why” and “how” of the system encoded in mathematical models to deduce more exactly what tomorrow will bring: how much light will shine on the solar panels or even when to shut the roof on Wimbledon to avoid the rain.

With climate change, the need for strong predictive structures become even more dire. Humanity is taking a complex, intricate system and dramatically altering a key component – greenhouse gases. We really are putting a cat amongst the pigeons, and, without doubt, things are going to change. The future is not going to be like the past: how it will change and how we can best avoid the worst–those are questions that require a mathematical and physical mastery of the system.

And of course, this is desperately important. While we in Britain might have enough resources and padding to quickly adapt to these changes, a lot of the people who share the earth with us are in a much more precarious position. Think of what famine does to a country with ethnic tensions, what water shortages in a country with a strong military imply for its neighbors and how a poor country deals with a new set of diseases traveling in with the weather.

Realizing the patterns, quantifying the interactions and building models is not a fix by itself. We still have the problem of wanting ever more growth and energy at as low a price as possible, of prioritizing today over generations from now.

But what we, as mathematical and physical thinkers, can contribute is a demystification: revealing the behavior of the complex Planet Earth so that as a society, a species, we can make the large ethical decisions facing us with more determination and confidence.

Mathematics, physical, geological, biological, ecological and chemical sciences don’t offer a pre-packaged answers: they’re ways of thinking to be drawn on depending on the question at hand. In this rest of this blog, we’ll be talking about some of the puzzles important in a responsible response to climate change and what tricks and elegant techniques from the mathematical world especially we can use. Expect calculus in the form of Partial Differential Equations used to describe a system changing over time, linear algebra to make those equations approximately solvable by a computer, dynamical systems thinking to try to simplify the complicated evolution and understand the butterfly effect and finally statistics and probability to express what we know and don’t know after all of that.