Category Archives: Mathematical Tools

What can Machine Learning do for Climate Science ?

Firstly, what is Machine Learning, other than a “sexy” buzzword used by the science community to make statistics sound cool? Well, per Andrew Ng, chief scientist at Baidu Inc. and an associate professor at Stanford it is “The science of getting computers to act without being programmed”. That statement leaves a lot to the imagination. One might start off by thinking of movies like I-Robot, Her and Ex Machina. For this post let’s stay grounded and say that Machine Learning explores the study and construction of algorithms that can learn from and make predictions from data. So, what can such an algorithm do for Climate Science? Turns out a lot.

 

Dr Claire Monteleoni, Assistance Professor at George Washington University, uses Machine Learning to track climate models. She along with many others are building a new field of Climate Informatics, a term she coined, with the aim of encouraging collaborations between climate scientists and Machine Learning researchers in order the bridge the gap between data and understanding. In her paper [1] Tracking Climate Models, she demonstrates the advantage of a Machine Learning approach for combining the predictions of multiple climate models. The Intergovernmental Panel on Climate Change currently use about 20 climate models to make informed decisions and predictions on climate change.  This paper introduces the use of an online learning algorithm called Learn-α to make predictions that match or surpass that of the best climate model.

data_learn_alpha

Machine Learning is also useful in application where there is little data or the data is sparse. For example, global maps of the partial pressure of CO2 (pCO2). Observations of sea surface pCO2 are taken mostly by commercial ships and consequently are sparse in both time and space, especially before the 1990s. Knowledge of this pCO2 is essential to investigate the variation of the ocean CO2 sink, from which data comparisons to the Global Carbon Budget can be made. To “fill in the gaps” Dr Peter Landschützer, from ETH Zürich, employed a Forward-Feed Neural Network. Neural Networks are based on the way the biological brain solves problems, using a large cluster of neurons connected by axons.

neurons-440660_1280

Finally, looking a bit closer to home, the Informatics Lab at the Met Office are applying Machine Learning techniques to traffic cameras. They are currently undertaking a project which will use data taken from traffic cameras to train a machine to recognize the weather. This is especially useful when considering snow. Snow is the hardest weather to forecast as it depends on small differences in pressure, temperature and heights of clouds. To know if it’s snowy it’s much easier to look at the ground. From the images on the traffic cameras the amount of white could tell you and furthermore characterise the snowy weather. What’s particularly cool about this project is that all the code and data is freely available on the Informatics Lab website http://www.informaticslab.co.uk

country-road-946436_1920

To conclude, we can see many applications of Machine Learning from combining the strengths of different Climate models to just wondering if it’s snowing.  The field of Climate Informatics is without doubt exciting and becoming more and more important. We might be a long way off a computer feeling cold but until then let’s use it to tell us more about our complicated climate.

 

[1] Monteleoni, C., Saroha, S., Schmidt, G., and Asplund, E.: Tracking Climate Models, Journal of Statistical Analysis and Data Mining, 4, 372–392, 2011.

More Than Just Symbols

 

When you think of a mathematics textbook you probably imagine a series of intimidating pages with a few words and a bunch of strange (often Greek) symbols. I don’t think I’m alone in thinking that the fact that a lot of modern mathematics is only presented in this form is a bit of a crime. For instance, Tristan Needham expresses a similar feeling in the pre-amble to his text “Visual Complex analysis.” Professional mathematicians can usually get some idea of what is going on in the pages of a paper or textbook. However, anyone who hasn’t had as much training in their past loses out. Particularly since, to the untrained eye, there is no way to associate these abstract symbols with anything visual or otherwise.

Reading a mathematics textbook is not like reading a novel–it can be a slow and arduous process. Despite this, someone with mathematical knowledge might eventually be able to understand what is going on. To draw an analogy with computers – it is as if you need the “correct software” installed in your mind to process the text. The same concept applies to reading novels written in other languages – if you don’t have the “correct software” installed in your mind then all that you will see is a series of random symbols. An education in mathematics allows you obtain this “software”, once you have this you can “speak” about things you have never spoken about before.

The important thing to draw from above is that the symbols are simply placeholders for various ideas and concepts just as they are in any other written language. In my opinion adding visuals or graphics to a piece of mathematics significantly helps us to tie down what the symbols are trying to suggest (even if it is just a crude analogy). On the other hand, it is probably true that even illustrations and graphs on their own are probably not enough. Without the proper context, graphs or illustrations may simply appear as static creations with no further meaning. I think additional understanding can be achieved by playing around with the image in your head (or by sketching variations with a pen and paper). This playful approach to imagining visualising mathematical concepts no doubt inspired artists such as M. C. Escher (M. C Escher was a Dutch graphic artist who is well known for his often mind boggling and mathematically inspired work).

Nowadays we can go significantly further than Escher with the power of computer graphics. As an example I have listed a few of my the coolest looking pages and blogs related to visualising mathematics and mathematical concepts below:

  • http://www.graphonaute.fr/ A selection of animations and images created by French engineering student Hugo Germain. Makes me think of what Escher’s work may have been like if he were born into the digital age!
  • http://blog.matthen.com/ Lots of nice visualisations and a few cool visual proofs of well known mathematical theorems as well. The code for the visualisations is also available allowing anyone to play around/ learn how to create their own!
  • http://imaginary.org/ Imaginary is an interactive platform which designed to showcase mathematical media content. The site contains plenty of pictures, videos and interactive demonstrations!
  • http://visualizingmath.tumblr.com/ A tumblr page full of math related visualisations.
  • http://geometric-aesthetic.tumblr.com/archive Tons of geometrical patterns and fractals.

In summary, the “beauty” of mathematics may be something “cold and austere” (as Bertrand Russell puts it), however, I believe everyone can gain if we do more to visualize the concepts involved. As well as helping our understanding, it allows us to think up strange new worlds (such as those depicted in other M.C Escher’s work – and potentially Einstein’s general relativity). Given the amount of maths out there I’m sure there is a lot of potential for mathematically inclined artists out there!

Matt
References and Further Reading
[1] This post is a continuation on themes I previously wrote about in about Maths and Visualization

Cover Image:

http://i.vimeocdn.com/video/458881089_1280x720.jpg

Programming is a piece of cake

When I think about famous mathematicians born in previous centuries, I imagine a handsome young man (well, my gender-biased imagination probably confirms the statements from my last article). He is sitting under a blooming tree with a quill and a scroll of parchment covered with equations. I must have lumped together mathematicians and poets, with the romantic charm.

With this image in mind, eighteen-year-old Paula went to her first lecture at the Faculty of Mathematics, Informatics and Mechanics (University of Warsaw). Full of expectations that I would do great things in the field of mathematics, I started copying down the words from the blackboard. “Baking a cake – the algorithm” – this is what they said. Was that a joke? No, just Introduction to Informatics.

You might wonder why such a course was obligatory for first year mathematics students. We just need a pen, a piece of paper and a great mind, right? Not quite. We can stay away from computers if we want to focus on abstract algebra or number theory. However, things get much more complicated if we would prefer to actually apply maths to other sciences. Like to weather prediction.

Lewis Fry Richardson tried to predict the weather on 20 May 1910 by hand. Don’t get me wrong, I admire his attempt, he was one of the pioneers of the weather prediction. Having said that, I am happy that scientists have developed more sophisticated techniques; Richardson failed quite badly.

Other examples? The model I am working on involves simulating the behaviour of population of worms. Without a computer I would have to give up, it is just impossible to calculate by a human being, not to mention more complicated systems, such as climate models.

The world of Four Colour Theorem

Computers even help us prove actual theorems from pure maths, for example the famous Four Colour Theorem. One of versions of this theorem states that any political map can be coloured using only four colours (or even less). The rule is that no bordering countries can have the same colour. The statement is very simple but the only way to prove it was to find all the possible configurations of countries (we are not talking about the existing countries but every map you could draw). I am pretty sure that without computers authors of the proof, Kenneth Appel and Wolfgang Haken, would be still checking all the possibilities.

I hope you understand now why the computers are so essential in mathematicians’ work. But what do they actually do for us and what does the cake mentioned in my first lecture have to do with maths?

Computers are like dogs and programming is just instructing the puppy to sit/stay/stop-stealing-my-favourite-socks-so-that-I-have-to-chase-you. You must know what you are given (input or your dog’s personality) and what you want to achieve (output or yay-I-can-wear-my-lovely-blue-socks). Anything in between is called the algorithm. Believe me or not, you are using algorithms every day. A recipe to bake a cake is one of them. Or your morning routine. When you forget one of the points or switch the order, you might have to eat a slack-baked cake or be late at work. Algorithms structure how we think about the problem.

How do we tell the computer what it is supposed to do? First we need to know in which language we are comfortable communicating. We can learn programming languages in similar way as we learn German or Spanish. The all have specific words, syntax etc. It takes practice to become fluent in one of them. And even if you are almost a native, you still need to consult a dictionary once in a while.

The problem with programming is that the computers are, paradoxically, very stupid. This is my personal view and I hope my laptop won’t take revenge on me for saying this (I still have to finish my project!). Why do I think so? If you confuse words in a foreign language, there are good chances that you will still be understood (unless you ask someone to turn you on, confusing it with turn around, which happened to me once). Computers just take what you give them and do not think. If your code is not working, it is entirely your fault. They will not try to guess what you wanted to do — do not count on their empathy. Life is brutal (and computers definitely are).

The first time you see a code – the instructions for the computer – you might be very surprised. It took me a while to accept that it is ok to write “a=a+1”. It means that we add 1 to a variable called “a”. Although if you accept “taking something with a pinch of salt”, why would you struggle with programming? We do not have to take everything literally!

I definitely think programming is a useful skill to acquire – and an essential one when you are a mathematician or a scientist. It can be hard, it can be exhausting. But trust me, the day when I managed to persuade my computer that I know what I want – and when I got it – was one of the best moments in my life. So keep calm and hello world!

 

Why studying random dynamical systems matters

If I said I was studying multistability in random dynamical systems driven by Brownian motion, why should you care?

Martingales, filtrations, ergodicity, synchronisation, Monte Carlo methods: mathematics is full of words which invoke colorful ephemeral imagery for heavy, abstract concepts. Even if we explained the concepts and brought them to life, it wouldn’t be obvious how they were related to your world. Mathematics is powerful exactly because it exists a few steps away from reality, in a place where anything which obeys its structured rules is possible. But this means that to answer the question of why we study what we study (apart from that it’s beautiful and fun) we have to step out of mathematics and back firmly into the physical.

When we think about the Planet Earth mathematically we often think about understanding and capturing each of its parts, studying technical topics like turbulence, boundary currents, atmospheric waves, sub-gridscale processes or atmospheric chemistry. These are the building blocks of the Earth and investigating them shows us patterns and connections which let us project their behaviour forward a few days into the future by running them artificially faster inside our super-computers, predicting the weather or the dissipation of volcanic ash.

There is another powerful way to think about our Earth, though. For the past decade or so, biologists, for example, have stepped away from the reductionist  approach of looking at the components of their system (protein and DNA) to start looking at the high-level systems behavior. The pieces slot together intricately into such a complex system that some of its behavior (consciousness, for an extreme example, or an ant colony) is nearly impossible to discern by looking at the parts. They find that sometimes it’s better to drop the details and start at the top and work down, characterising as much of the emergent behaviour as possible since it’s part of understanding the animal, even if we can’t quite see where it comes from.

In the study of Planet Earth we’re lucky that we’ve been so driven early on by prediction even while we’ve been studying the pieces. We’ve always had an eye on the whole system, trying to simulate it in weather models since predictability is equivalent to safety and success as we try to flourish as a species. These models do an amazing job of mirroring the massive weather beast, especially as we tweak them year on year, checking them against reality and improving them. But what we don’t often do is explore theoretically the range of behaviour the complicated highly non-linear and multi-dimensional system we live within is capable of. We don’t often look at it from the top without the baggage of the details of the pieces. This is the truly “systems” approach.

Perhaps the simplest example of what you might see if you took a systems perspective is multistability: the idea that the Earth might have different states (or rhythms) it could comfortably fall into which would self-perpetuate themselves, even if in the same conditions another state would have been possible. The mathematical example is always a ball rolling down into a valley with another valley next door; the ball could stop at the bottom of either valley, both are stable states, but it can’t get from one to another easily. Ecosystems exhibit multi-stability – the same patch of land could be a birch forest or a grazed field and stay that way indefinitely. Even bodies exhibit multistability if you look at them from the right perspective: something alive generally keeps itself alive, but dead also stays dead.

A schematic view of bistability, in which the “ball” can be in either “valley” happily, but finds it difficult to transition between the two. Source: Wikimedia Commons.

Whether or not our climate has multiple stable states is vitally important because if it does and we start to push it, we could very possibly push it out of one and into another and it might be very hard to return. How would we know if it did? One way (the reductionist way) might be to carefully build a replica of it, to the very best of our ability, then start pushing it around and see if it switches. Not a bad idea, but with obvious drawbacks too: our model might not be quite perfect and it takes a long time to run these pushing experiments on a computer, so it’d be hard to draw solid conclusions. Another way is to think theoretically about what is required of a system to be a jumping system then study whether the Earth satisfies the conditions, looking back at the record we have about our real Earth system to see if we can see signatures of jumping. To do that, however, we need to know what jumping looks like.

That’s why we’re interested in a class of mathematical constructs called random dynamical systems. They’re an abstract construct and could be interpreted into the real world in lots of different ways so, like most everything in mathematics, they’re probably useful in a lot of fields. What they can do for us, though, is to represent the Earth like a point moving between states according to some update rule, complicated or simple, which summarises the evolution of our climate in time. What makes them even more complicated than the dynamical systems people like Ed Lorenz have been studying for decades is that we now allow the update rule to be a little bit random thanks to a Brownian motion term: to send the Earth up the valley when it should have gone down, occasionally. Suddenly, a whole new class of behaviour is exposed: with bistabilitity in the update scheme and additional noise comes stochastic resonance, the seemingly regular jumping of our proverbial ball between valleys depending on which way the wind (which wouldn’t usually be enough to move a ball between valleys!) blows. They also might simplify dynamical systems, with noise-induced synchronisation meaning that even if we don’t know where exactly the point started, it’ll end up in a similar place if it undergoes the same noise.

It’s not that the Earth is a random dynamical system (it’s a planet, not an abstract mathematical concept), it’s that we hope that by trying to represent it as one, we’ll learn more about it. Climate certainly has some random elements: weather provides unpredictable and sometimes large change in local climate, not to mention volcanoes. We also have stochastic resonance-looking jump-like events recorded in the history: the Glacial-Interglacials and maybe Dansgaard-Oeschger events in the past million years. If we find that to some extent our reality fits well into this concept, we will be able to draw conclusions from the concept back out to the real world. It probably won’t be as revealing as complex wave functions were for quantum mechanics, but having a new way to account for the world around us is almost surely a good thing.

When it comes to complex systems that we count on for our survival, no one approach can be relied on. We don’t have a reason to believe that there’s a single bit of mathematics that nature obeys on this scale, but there will be patterns and properties at all scales which fit into mathematical frameworks. Random dynamical systems let us look at the pattern of stability and large changes in the whole system and is, as both a mathematical field and a climate perspective, relatively new and so very exciting.

When the mathematicians meet…

What’s the difference between an introverted mathematician and an extroverted mathematician? The extrovert looks at the other person’s shoes. You probably have heard this joke (very funny) many times and might believe that mathematicians work stuck in their offices (and their own heads). It was probably true a couple of centuries ago. However, things have changed after the rapid development of maths, especially applied maths. Why?

Nowadays we have to specialise, at least a little bit. We don’t have Da Vinci’s any more; it’s just impossible to fully understand more than your very narrow area. But while mathematics research narrows more and more, it tries to tackle more and more complex and multidisciplinary problems. What do we do now?!

We, as mathematicians, must get out of our comfort zones and collaborate. We need to accept our lack of understanding of certain aspects of each maths problem while, at the same time, being aware of how we can contribute to the solution. We have to identify what kind of experts we need to ask for help to make some progress. This is how it all begins.

When I started my adventure in maths, I certainly didn’t anticipate this. I was prepared for working alone and talking to colleagues only in my free time. While it could possibly work in pure maths, I would totally fail to succeed in applied fields if I tried to do so.

Examples? Mathematics of Planet Earth Centre for Doctoral Training! Yes, we pursue our individual degrees and work on our own projects. However, we operate as a cohort too. Sharing experience, tips and asking for help are essential for this programme to exist. I can’t even count how many times computer science experts saved my life (or at least my precious laptop) by preventing me from running a code that would destroy the system1 . In exchange I could give them a hand when they got lost in abstract multidimensional spaces (although I don’t claim I can visualise anything in more than three dimensions, though it disappoints my first year lecturer!). We all learn from one another.

Recently I realised that it’s not just a fake academic set-up, this is how the “real world mathematics“ works. I spent 5 days at the 116th European Study Group with Industry in Durham, UK (http://www.esgi.org.uk/). This event brought together about hundred mathematicians, physicists and industrial partners. The latter proposed eight problems they wanted to solve in fields as diverse as agriculture, banking and sepsis diagnosis. We divided ourselves into groups according to our interests — I chose the problem proposed by a digital bank. They needed help with marketing their product to the best target audience (of course the ultimate goal was to spend less and earn more). We sat down in a room and…well, and started thinking, talking, brainstorming and arguing. Within 3 days we managed to produce whole models and get some useful results for the industrial partners. Something infeasible for one genius became a reality for a group of people with different backgrounds.

Yes, you might still meet a mathematician staring at her/his own shoes while talking (or avoiding any contact) to you. But this is not a norm anymore. And definitely not the only way to succeed. We can tackle real world problems together because together impossible becomes possible!

  1. Note, not everyone is as lucky as me, you can read what happens when you’re not careful while programming here.

What on Earth is Mathematics of Planet Earth?

Until recently I had no problem explaining what I was studying; I was just an average maths student. I could reliably predict the reaction of a person informed of this fact: “How can you do that? I’ve always been hopeless at maths. And anyway, what are you going to end up doing with that degree”? Things got much more complicated when I started a PhD at Mathematics of Planet Earth. The first reaction is now usually the question used as a title to this article: what on earth is mathematics of planet earth?! After a brief explanation that I am learning how to use and develop maths for climate and weather predictions, I just get a reassuring statement: “I know that this whole climate change thing is very dangerous/rubbish” (choose the option that applies to you) and a question: “But why did you resign from doing proper maths?”

Actually, I am more involved in studying mathematics than ever. No science would exist without mathematics, in particular climatology or meteorology. Some people can predict the rain by feeling it in their bones; I can “predict” the rain more or less based on the fact that we are in UK. But do we really want to risk our life on someone’s body niggles? No, I do not exaggerate. Our life really can depend on it. Do not forget that a bad weather prediction not only can get you wet, but also farmers might not prepare for a drought (and the crops would get extremely expensive next year), local authorities might not decide to grit ice-covered roads (so you might get stuck in traffic or even have an accident) or a dangerous storm might hit citizens completely not ready for it. It is something worth looking at, is it not?

To get more reliable predictions about the state of atmosphere in the next couple of hours, days or even centuries, we need… mathematics. No, not that boring multiplication table, but nearly every field of very advanced mathematics. Let us take a look at a couple of examples.

Chaos Theory

You will have heard of the “butterfly effect” which allegedly can provoke a hurricane. This is all about the chaos. The intuitive definition, given by E. Lorenz, the creator of chaos theory, is [1] when the present determines the future, but the approximate present does not approximately determine the future. It means that if we were given infinitely precise initial conditions (i.e. full description of the state of the weather now), we could predict the weather at any time in the future. So why do meteorologists sometimes get it wrong? Because this is just wishful thinking. In reality we are not able to get perfect measures of the weather components, for example due to the limitations of measuring devices. Thus mathematicians need to choose the most important measurements with the available precision and try to get the best prediction they can. However, chaos theory states that, under some conditions, starting from almost the same state we can get completely different results. It complicates weather prediction so chaos theory is still something we need to study.

Numerical Analysis

There would be no weather forecast without very advanced computers we are using. Some of them are even supercomputers, such as the one used by Met Office. It costed a trifling £97 million. Why do governments invest such enormous sums into such equipment? Before we understand that, we have to see how the weather prediction works. As mentioned above, we cannot forecast it exactly. Hence mathematicians have to get rid of some parameters that seem to be less important (by the way, deciding which are those is far from obvious) and, using the ones that are left, build a model. This is a set of equations (sometimes thousands of them!) that describe the system. Do you remember solving systems of two equations at school? You might have struggled with it. So now imagine solving thousands of much more complicated ones. Yes, this is exactly why we need supercomputers; they make this job feasible. However, mathematicians still need to make sure that the result produced by a computer is sensible. They do it by carrying out a numerical analysis, checking the properties of the system.

I’ve mentioned only a tiny fraction of the whole range of mathematical tools used in the weather prediction. Next time when you listen to your favourite weather forecast, keep in mind that it would not make any sense without mathematics. And if you happen to have a child, encourage them to study maths. Just in case.

 

[1] Danforth, Christopher M. (April 2013). “Chaos in an Atmosphere Hanging on a Wall”. Mathematics of Planet Earth 2013. Retrieved 27 January 2016.