It looks like you're new here. If you want to get involved, click one of these buttons!

- All Categories 1.6K
- Azimuth Code Project 107
- News and Information 342
- Chat 199
- Azimuth Blog 148
- Azimuth Forum 29
- Azimuth Project 190
- - Strategy 109
- - Conventions and Policies 21
- - Questions 43
- Azimuth Wiki 704
- - Latest Changes 696
- - - Action 14
- - - Biodiversity 8
- - - Books 1
- - - Carbon 9
- - - Computational methods 38
- - - Climate 52
- - - Earth science 23
- - - Ecology 43
- - - Energy 29
- - - Experiments 30
- - - Geoengineering 0
- - - Mathematical methods 69
- - - Meta 9
- - - Methodology 16
- - - Natural resources 7
- - - Oceans 4
- - - Organizations 34
- - - People 6
- - - Publishing 3
- - - Reports 3
- - - Software 20
- - - Statistical methods 2
- - - Sustainability 4
- - - Things to do 2
- - - Visualisation 1
- General 36

Options

I started a page

to strengthen our section on climate concepts and also make a connection to Tim's interest in stochastic differential equations. There's a nice paper on using stochastic differential equations to model the irregularity in this climate pattern, which I would like to talk about on the Azimuth Project.

## Comments

Sidenote: I'm not interested in SDE themselves, but as a tool for modelling in climate and earth sciences - it's just me who needs to refresh the math first, before looking at the applications. In the meantime pages like SDE or Wavelet will look somewhat out of place.

`Sidenote: I'm not interested in SDE themselves, but as a tool for modelling in climate and earth sciences - it's just me who needs to refresh the math first, before looking at the applications. In the meantime pages like SDE or [[Wavelet]] will look somewhat out of place.`

Tim wrote:

Okay, good! I just thought I'd learn a bit about applications of SDE and put some of that information into the SDE page as I do. I just happen to have Tim Palmer's book

Stochastic Physics and Climate Modellingchecked out from the library, since an interview with Palmer will probably appear in the next This Week's Finds. He's a big advocate of stochastic methods in climate modelling.`Tim wrote: >Sidenote: I'm not interested in SDE themselves, but as a tool for modelling in climate and earth sciences - it's just me who needs to refresh the math first, before looking at the applications. Okay, good! I just thought I'd learn a bit about applications of SDE and put some of that information into the [[SDE]] page as I do. I just happen to have Tim Palmer's book _Stochastic Physics and Climate Modelling_ checked out from the library, since an interview with Palmer will probably appear in the next This Week's Finds. He's a big advocate of stochastic methods in climate modelling.`

Is Tim Palmer's book a good introduction to the subject? I don't have one yet...It is this one, correct: amazon?

Of course I could try to check it out from the Bayerische Staatsbibliothek, that is thankfully open to everyone. (When I was there to get a library card I was asked thrice: "You

do knowthat you can't get any movies here, it's specialist literature only!".)`Is Tim Palmer's book a good introduction to the subject? I don't have one yet...It is this one, correct: <a href="http://www.amazon.com/Stochastic-Physics-Climate-Modelling-Italian/dp/0521761050">amazon</a>? Of course I could try to check it out from the Bayerische Staatsbibliothek, that is thankfully open to everyone. (When I was there to get a library card I was asked thrice: "You <i>do know</i> that you can't get any movies here, it's specialist literature only!".)`

Tim wrote:

That's the book.

Whether it's a good introduction to "the subject" depends strongly on what "the subject" is. It's not an introduction to climate modelling, which I believe usually does

notuse stochastic methods. It's a collection of papers by various people whodoapply stochastic methods to climate modelling. So, you'll see things like a paper on "Challenges in stochastic modelling of quasi-geostrophic turbulence" that does not define "quasi-geostrophic turbulence". On the other hand, I don't know any other book that discusses stochastic methods in climate modelling.So, you should probably take a look at it before buying it.

Are you trying to learn about climate modelling in general, or trying to find some models that involve stochastic differential equations? What are you dreaming of, exactly? I'm curious, since I'd like to help.

I do recommend the paper by Vallis, "Mechanisms of climate variability from years to decades", that I linked to on the SDE page. It's free, it's good, and it's stochastic!

`Tim wrote: >Is Tim Palmer's book a good introduction to the subject? I don't have one yet...It is this one, correct: <a href="http://www.amazon.com/Stochastic-Physics-Climate-Modelling-Italian/dp/0521761050">amazon</a>? That's the book. Whether it's a good introduction to "the subject" depends strongly on what "the subject" is. It's not an introduction to climate modelling, which I believe usually does _not_ use stochastic methods. It's a collection of papers by various people who _do_ apply stochastic methods to climate modelling. So, you'll see things like a paper on "Challenges in stochastic modelling of quasi-geostrophic turbulence" that does not define "quasi-geostrophic turbulence". On the other hand, I don't know any other book that discusses stochastic methods in climate modelling. So, you should probably take a look at it before buying it. Are you trying to learn about climate modelling in general, or trying to find some models that involve stochastic differential equations? What are you dreaming of, exactly? I'm curious, since I'd like to help. I do recommend the paper by Vallis, "Mechanisms of climate variability from years to decades", that I linked to on the [[SDE]] page. It's free, it's good, and it's stochastic!`

I'm learning about climate models from the "Climate Modelling Primer" that Nathan recommended, which is exactly at the level I need right now. But I'm curious: I thought that stochastic methods do

naturallyplay an important role in weather and climate modelling, but it would seem that this is not so.Why did I think that? Well, first of all I was trained as a physicist, so I naturally try to model big complicated systems using statistical physics :-) Secondly, I must have had some association when I heard "chaos and butterfly effect". The "butterfly effect" is IMHO a huge misunderstanding: Sure, a small perturbation of the initial conditions of a system of ODE has an arbitrary big impact on the system, if by "impact" we think about the quality of an approximation using the unperturbed initial conditions, at an arbitrary late time. But that's not applicable to the weather, because it is a stochastic system, so that a system of ODE is not a good model, and in the real world the effect of a butterfly gets lost completely in the noise of other influences.

I'm like the complete stranger stumbling into a foreign country wondering why people do the things the way they do :-) (You know, like "why do the Americans build their houses of wood, when there are so many natural phenomena like hurricanes that easily destroy such constructions? Germans build all their houses using bricks, although there never are any hurricanes. That is insane, it should be the other way 'round!").

`<blockquote> <p> Are you trying to learn about climate modelling in general, or trying to find some models that involve stochastic differential equations? What are you dreaming of, exactly? </p> </blockquote> I'm learning about climate models from the "Climate Modelling Primer" that Nathan recommended, which is exactly at the level I need right now. But I'm curious: I thought that stochastic methods do <i>naturally</i> play an important role in weather and climate modelling, but it would seem that this is not so. Why did I think that? Well, first of all I was trained as a physicist, so I naturally try to model big complicated systems using statistical physics :-) Secondly, I must have had some association when I heard "chaos and butterfly effect". The "butterfly effect" is IMHO a huge misunderstanding: Sure, a small perturbation of the initial conditions of a system of ODE has an arbitrary big impact on the system, if by "impact" we think about the quality of an approximation using the unperturbed initial conditions, at an arbitrary late time. But that's not applicable to the weather, because it is a stochastic system, so that a system of ODE is not a good model, and in the real world the effect of a butterfly gets lost completely in the noise of other influences. I'm like the complete stranger stumbling into a foreign country wondering why people do the things the way they do :-) (You know, like "why do the Americans build their houses of wood, when there are so many natural phenomena like hurricanes that easily destroy such constructions? Germans build all their houses using bricks, although there never are any hurricanes. That is insane, it should be the other way 'round!").`

Tim writes:

That's a very good state to be in, if you want to make interesting discoveries. Of course sometimes people have good but nonobvious reasons to do what they do, which you never thought of before. But other times they simply haven't thought about doing them any other way. And either way, you have an opportunity to improve things. Information can flow in both directions.

For example, someday German hotels may discover washcloths, and the idea that several thin blankets on a bed are more useful than a single incredibly thick down-filled comforter that makes you sweat like a pig when you put it on and freeze when you take it off. On the other hand, someday American hotels may discover the idea of serving healthy and tasty breakfasts as German hotels often do.

As I've said before, Tim Palmer seems to be the main advocate of stochastic methods in climate modelling. I get the feeling that the general idea is: "everything is so incredibly complicated and uncertain already, why should we deliberately make our models more complicated and uncertain by deliberately adding random noise?" And this is not, in fact, a completely stupid objection.

I'm not sure I completely agree with you here: some of the stochasticity in the weather is in fact

dueto the amplification of small, unmodeled hence "random" perturbations in the initial data. However, instead of arguing (since I'm too ignorant to win the argument), I'd prefer to recommend that you read another volume edited by Tim Palmer and Renate Hagedorn:Predictability of Weather and Climate. It's all about exactly these issues: stochasticity versus chaos, the "butterfly effect" versus "therealbutterfly effect", etc.`Tim writes: >I'm like the complete stranger stumbling into a foreign country wondering why people do the things the way they do That's a very good state to be in, if you want to make interesting discoveries. Of course sometimes people have good but nonobvious reasons to do what they do, which you never thought of before. But other times they simply haven't thought about doing them any other way. And either way, you have an opportunity to improve things. Information can flow in both directions. For example, someday German hotels may discover washcloths, and the idea that several thin blankets on a bed are more useful than a single incredibly thick down-filled comforter that makes you sweat like a pig when you put it on and freeze when you take it off. On the other hand, someday American hotels may discover the idea of serving healthy and tasty breakfasts as German hotels often do. As I've said before, Tim Palmer seems to be the main advocate of stochastic methods in climate modelling. I get the feeling that the general idea is: "everything is so incredibly complicated and uncertain already, why should we deliberately make our models more complicated and uncertain by deliberately adding random noise?" And this is not, in fact, a completely stupid objection. > The "butterfly effect" is IMHO a huge misunderstanding: Sure, a small perturbation of the initial conditions of a system of ODE has an arbitrary big impact on the system, if by "impact" we think about the quality of an approximation using the unperturbed initial conditions, at an arbitrary late time. But that's not applicable to the weather, because it is a stochastic system... I'm not sure I completely agree with you here: some of the stochasticity in the weather is in fact *due* to the amplification of small, unmodeled hence "random" perturbations in the initial data. However, instead of arguing (since I'm too ignorant to win the argument), I'd prefer to recommend that you read another volume edited by Tim Palmer and Renate Hagedorn: _[Predictability of Weather and Climate](http://www.amazon.com/Predictability-Weather-Climate-Tim-Palmer/dp/0521848822)_. It's all about exactly these issues: stochasticity versus chaos, the "butterfly effect" versus "the _real_ butterfly effect", etc.`

This is probably naive, but I'd have thought that a big part of the reason would have been a belief (whether justified or not) that at the level of climatological "variables" extra random noise would "average out" so that there's no real difference in the result. (Obviously that's a very dubious belief if you think your mathematical system has bifurcations/tipping points/mulitple attractors.)

`> "everything is so incredibly complicated and uncertain already, why should we deliberately make our models more complicated and uncertain by deliberately adding random noise?" This is probably naive, but I'd have thought that a big part of the reason would have been a belief (whether justified or not) that at the level of climatological "variables" extra random noise would "average out" so that there's no real difference in the result. (Obviously that's a very dubious belief if you think your mathematical system has bifurcations/tipping points/mulitple attractors.)`

John said

(Grinning:) The trick is a sleeping style where you stick out one leg and tuck it in and stick out the other once the first one gets too cold :-)

The German hotels I know offer breakfast and

continentalbreakfast, the first being what normal people eat in the morning, you know, the British: Eggs & bacon & sausages etc.And why can't I get a decent croissant in Germany?! (You spend some time in Paris, did you get to love those?)

Of course I'm not trying to convince anyone of anything, I'm just trying to explain my own surprise to myself.

That's probably the best way to proceed.

Sure: I tend to assume that there are/is some

hiddenassumptions or knowledge that I don't know about, and I try to provoke others to explain it to me by stating my surprise :-)(Although I have to admit that there are situations where I use the word "stupid" and mean it.)

Now about stochastic physics and climate models:

Of course the models already are stochastic, because the equations they use describe macroscopic statistical relations (like the ideal gas).

The authors of the "primer" confirm that "ensemble runs" of models are commen, just like Nathan explained: Assume some probability distribution on your input parameters, sample from it and calculate the resulting distribution of model results. This, of course, is stochastic, too.

Using

stochasticODE or PDE simply increases your model portfolio.The authors of the "primer" mention that "some climate modelling groups seem to try to model every molecule individually": Now, that's irony, of course, but - taking it at face value, for the moment - as a physicist I'd say: Statistical physics teaches us that this kind of modelling is not possible and not necessary - and of course would go awfully wrong because the more detailled you model the more errors you'll make, especially if you don't use quantum mechanics :-) Introducing stochastic elements actually

simplifiesmatters.I'll report on how my POV has changed after reading "Predictability of Weather and Climate."

P.S.: Is it an idea or already a worked out plan to interview Tim Palmer on the next TWF?

`John said <blockquote> <p> ...several thin blankets on a bed are more useful than a single incredibly thick down-filled comforter that makes you sweat like a pig when you put it on and freeze when you take it off. </p> </blockquote> (Grinning:) The trick is a sleeping style where you stick out one leg and tuck it in and stick out the other once the first one gets too cold :-) <blockquote> <p> On the other hand, someday American hotels may discover the idea of serving healthy and tasty breakfasts as German hotels often do. </p> </blockquote> The German hotels I know offer breakfast and <i>continental</i> breakfast, the first being what normal people eat in the morning, you know, the British: Eggs & bacon & sausages etc. And why can't I get a decent croissant in Germany?! (You spend some time in Paris, did you get to love those?) <blockquote> <p> I'm not sure I completely agree with you here: some of the stochasticity in the weather is in fact due to the amplification of small, unmodeled hence "random" perturbations in the initial data. </p> </blockquote> Of course I'm not trying to convince anyone of anything, I'm just trying to explain my own surprise to myself. <blockquote> <p> I'd prefer to recommend that you read another volume edited by Tim Palmer and Renate Hagedorn: Predictability of Weather and Climate. </p> </blockquote> That's probably the best way to proceed. <blockquote> <p> "everything is so incredibly complicated and uncertain already, why should we deliberately make our models more complicated and uncertain by deliberately adding random noise?" And this is not, in fact, a completely stupid objection. </p> </blockquote> Sure: I tend to assume that there are/is some <i>hidden</i> assumptions or knowledge that I don't know about, and I try to provoke others to explain it to me by stating my surprise :-) (Although I have to admit that there are situations where I use the word "stupid" and mean it.) Now about stochastic physics and climate models: 1. Of course the models already are stochastic, because the equations they use describe macroscopic statistical relations (like the ideal gas). 2. The authors of the "primer" confirm that "ensemble runs" of models are commen, just like Nathan explained: Assume some probability distribution on your input parameters, sample from it and calculate the resulting distribution of model results. This, of course, is stochastic, too. 3. Using <i>stochastic</i> ODE or PDE simply increases your model portfolio. 4. The authors of the "primer" mention that "some climate modelling groups seem to try to model every molecule individually": Now, that's irony, of course, but - taking it at face value, for the moment - as a physicist I'd say: Statistical physics teaches us that this kind of modelling is not possible and not necessary - and of course would go awfully wrong because the more detailled you model the more errors you'll make, especially if you don't use quantum mechanics :-) Introducing stochastic elements actually <i>simplifies</i> matters. I'll report on how my POV has changed after reading "Predictability of Weather and Climate." P.S.: Is it an idea or already a worked out plan to interview Tim Palmer on the next TWF?`

AOGCM modelers often run their models several times with different initial conditions, to get at the chaotic initial-value uncertainty in the system. But they don't usually inject stochastic noise

insidethe model. Each run is deterministic, given an initial condition. A stochastic representation of unresolved or parameterized processes might help. It is usually thought that such extra stochastic noise is irrelevant to long term climate prediction, but Tim Palmer has made the case that it may at least improve short-term predictive skill, when chaotic variability does matter.`AOGCM modelers often run their models several times with different initial conditions, to get at the chaotic initial-value uncertainty in the system. But they don't usually inject stochastic noise _inside_ the model. Each run is deterministic, given an initial condition. A stochastic representation of unresolved or parameterized processes might help. It is usually thought that such extra stochastic noise is irrelevant to long term climate prediction, but Tim Palmer has made the case that it may at least improve short-term predictive skill, when chaotic variability does matter.`

To clarify, when GCM modelers talk about an ensemble of runs, they're usually

nottalking about specifying a probability distribution of input parameters. They're just perturbing initial conditions. There are parametric "perturbed physics ensembles"of GCM runs, but there are very few (just the ClimatePrediction.net people, some limited work with NCAR's CAM, and a few others).`To clarify, when GCM modelers talk about an ensemble of runs, they're usually _not_ talking about specifying a probability distribution of input parameters. They're just perturbing initial conditions. There are parametric "perturbed physics ensembles"of GCM runs, but there are very few (just the ClimatePrediction.net people, some limited work with NCAR's CAM, and a few others).`

I see, I must have over-interpreted the word "ensemble" in "ensemble runs".

AFAIK varying the input parameters as well as varying the discretization parameters like the grid spacing is a standard procedure to test if the numerical solution is stable. But when you say...

...it seems that matters are a little more complicated than that (i.e. testing the model for numerical instabilities).

I'd like to know more about this aspect, (the primer (first half) talks about deterministic models only, too).

Is this assessment

based on experience/experiments?

a gut feeling based on a qualitative understanding of climate models

a line of defense against methods that cannot be implemented anyway because state-of-the-art models eat all computer power available anyway?

something else?

`I see, I must have over-interpreted the word "ensemble" in "ensemble runs". AFAIK varying the input parameters as well as varying the discretization parameters like the grid spacing is a standard procedure to test if the numerical solution is stable. But when you say... <blockquote> <p> AOGCM modelers often run their models several times with different initial conditions, to get at the chaotic initial-value uncertainty in the system. </p> </blockquote> ...it seems that matters are a little more complicated than that (i.e. testing the model for numerical instabilities). <blockquote> <p> A stochastic representation of unresolved or parameterized processes might help. It is usually thought that such extra stochastic noise is irrelevant to long term climate prediction... </p> </blockquote> I'd like to know more about this aspect, (the primer (first half) talks about deterministic models only, too). Is this assessment 1. based on experience/experiments? 2. a gut feeling based on a qualitative understanding of climate models 3. a line of defense against methods that cannot be implemented anyway because state-of-the-art models eat all computer power available anyway? 4. something else?`

I think you implied that models are run with different initial conditions to test for numerical instabilities. This isn't the main motivation. The main motivation is to determine the spread of "natural variability" in the predictions, due to initial value uncertainty.

As for your second question, it's usually some combination of 1 and 2. People could make stochastic climate models if they wanted to, and it wouldn't be that more computationally intensive. The general view is that for long term projections, what dominates is basic stock-flow dynamical arguments such as "energy balance". Stochastic noise tends to "fuzz" the projections out a bit, but doesn't usually alter the mean state.

There are some places where I'd expect this argument to break down. For temperature, it's pretty much just how much CO2 you put into the air. But for extreme weather events, or quasi-chaotic phenomena like ENSO, it could be important to have stochastic dynamics even for long term projections. One exception is if you're right on the edge of a tipping point, where extra noise will push you over the edge. But we probably won't end up finely balanced near a tipping point. We'll probably end up clearly on one side or the other, regardless of stochastic internal variability. I think stochastic variability for long term climate is more relevant to the glacial-interglacial cycles, where such fluctuations may drive some of the diversity of responses we see to the steady Milankovitch cycles.

Another possibility could be that stochastic variability could improve our representation of some sub-grid scale cloud processes which do propagate up to the macroscale, altering the overall strength of the cloud feedback. This could end up altering the projected mean state, not just the variability around it. This is rather speculative.

`I think you implied that models are run with different initial conditions to test for numerical instabilities. This isn't the main motivation. The main motivation is to determine the spread of "natural variability" in the predictions, due to initial value uncertainty. As for your second question, it's usually some combination of 1 and 2. People could make stochastic climate models if they wanted to, and it wouldn't be that more computationally intensive. The general view is that for long term projections, what dominates is basic stock-flow dynamical arguments such as "energy balance". Stochastic noise tends to "fuzz" the projections out a bit, but doesn't usually alter the mean state. There are some places where I'd expect this argument to break down. For temperature, it's pretty much just how much CO2 you put into the air. But for extreme weather events, or quasi-chaotic phenomena like ENSO, it could be important to have stochastic dynamics even for long term projections. One exception is if you're right on the edge of a tipping point, where extra noise will push you over the edge. But we probably won't end up finely balanced near a tipping point. We'll probably end up clearly on one side or the other, regardless of stochastic internal variability. I think stochastic variability for long term climate is more relevant to the glacial-interglacial cycles, where such fluctuations may drive some of the diversity of responses we see to the steady Milankovitch cycles. Another possibility could be that stochastic variability could improve our representation of some sub-grid scale cloud processes which do propagate up to the macroscale, altering the overall strength of the cloud feedback. This could end up altering the projected mean state, not just the variability around it. This is rather speculative.`

Tim wrote:

The interview was done months ago, and I just need to write some extra stuff to go around it. I'm also considering asking him a few more questions. Or maybe he will find time to answer some questions on Azimuth. He's a busy guy: he only found time to answer the questions so far after I created a volcanic eruption in Iceland, disrupting his travel plans.

After Tim Palmer, the next interviews in This Week's Finds will be with Eliezer Yudkowsky, Thomas Fischbacher and David Ellerman. These are also mostly done.

`Tim wrote: >P.S.: Is it an idea or already a worked out plan to interview Tim Palmer on the next TWF? The interview was done months ago, and I just need to write some extra stuff to go around it. I'm also considering asking him a few more questions. Or maybe he will find time to answer some questions on Azimuth. He's a busy guy: he only found time to answer the questions so far after I created a volcanic eruption in Iceland, disrupting his travel plans. After Tim Palmer, the next interviews in This Week's Finds will be with Eliezer Yudkowsky, Thomas Fischbacher and David Ellerman. These are also mostly done.`

Impressive! I had no idea of the amount of energy and work you've already put into Azimuth...

I knew it! Mathematical physics is this arcane art, that only special people with this "special" talent are able to learn, who have to pass a series of mysterious initiation rituals, being endorsed by elusive high priests of the art. They communicate in familiar yet foreign tongues, using never heard of runes and symbols...finally one of them steps forward and openly admits his supernatural powers!

`<blockquote> <p> After Tim Palmer, the next interviews in This Week's Finds will be with Eliezer Yudkowsky, Thomas Fischbacher and David Ellerman. These are also mostly done. </p> </blockquote> Impressive! I had no idea of the amount of energy and work you've already put into Azimuth... <blockquote> <p> ...after I created a volcanic eruption in Iceland...</p> </blockquote> I knew it! Mathematical physics is this arcane art, that only special people with this "special" talent are able to learn, who have to pass a series of mysterious initiation rituals, being endorsed by elusive high priests of the art. They communicate in familiar yet foreign tongues, using never heard of runes and symbols...finally one of them steps forward and openly admits his supernatural powers!`

I vastly improved the introductory section in

ENSO

by rewriting the description of this cycle in a way that's easier to understand, including an animation that shows how the cycle works, and adding two graphs that illustrate the El Niño-La Niña cycle and the related Southern Oscillation.

I'm writing a This Week's Finds, "week307", that includes some stuff about climate cycles. So, I'll be improving a few articles on this subject.

`I vastly improved the introductory section in [[ENSO]] by rewriting the description of this cycle in a way that's easier to understand, including an animation that shows how the cycle works, and adding two graphs that illustrate the El Niño-La Niña cycle and the related Southern Oscillation. I'm writing a This Week's Finds, "week307", that includes some stuff about climate cycles. So, I'll be improving a few articles on this subject.`

I added a section on "Modelling the ENSO" to

ENSO

and also added more to "Stochastic aspects". First I talk about some equations that describe a Hopf bifurcation; then I talk about adding white noise to these equations. All this is discussed in this paper, which is free online:

Stochastic Physics and Climate Modelling, eds. Tim Palmer and Paul Williams, Cambridge U. Press, Cambridge, 2010.It seems to me that fairly small changes in the program we use to illustrate stochastic resonance could yield a program that would illustrate the Hopf-bifurcation-with-noise-added! The main difference is that the equation describes two functions of time instead of one, and the differential equation is time-independent rather than explicitly time-dependent.

I would love it if one of you programming experts could make these changes in the program Tim already wrote! If you teach me enough I could do it myself, but it might be more efficient if I find and explain nice toy climate models, and you guys write the programs. Does that make sense?

I don't want you to feel like I'm pushing you to do work you'd rather not do. I'm hoping that you'd find it fun. I'm hoping I could talk about these models in

This Week's Finds, include pretty pictures or even animations made by you guys, give you full credit for the software, point readers to the software, and use all this to advertise the Azimuth Code Project.I'll start by doing this for stochastic resonance, where Tim has already done most or all of the programming work.

By the way, there's also a more realistic ENSO model called the Zebiak–Cane model. It's supposed to be the simplest model that exhibits fairly realistic ENSO behavior. I included some references. But I don't know what it looks like yet!

`I added a section on "Modelling the ENSO" to [[ENSO]] and also added more to "Stochastic aspects". First I talk about some equations that describe a Hopf bifurcation; then I talk about adding white noise to these equations. All this is discussed in this paper, which is free online: * Richard Kleeman, [Stochastic theories for the irregularity of the ENSO](http://www.pims.math.ca/files/kleeman_6.pdf), in _Stochastic Physics and Climate Modelling_, eds. Tim Palmer and Paul Williams, Cambridge U. Press, Cambridge, 2010. It seems to me that fairly small changes in the program we use to illustrate [[stochastic resonance]] could yield a program that would illustrate the Hopf-bifurcation-with-noise-added! The main difference is that the equation describes two functions of time instead of one, and the differential equation is time-independent rather than explicitly time-dependent. I would love it if one of you programming experts could make these changes in the program Tim already wrote! If you teach me enough I could do it myself, but it might be more efficient if I find and explain nice toy climate models, and you guys write the programs. Does that make sense? I don't want you to feel like I'm pushing you to do work you'd rather not do. I'm hoping that you'd find it fun. I'm hoping I could talk about these models in _This Week's Finds_, include pretty pictures or even animations made by you guys, give you full credit for the software, point readers to the software, and use all this to advertise the Azimuth Code Project. I'll start by doing this for [[stochastic resonance]], where Tim has already done most or all of the programming work. By the way, there's also a more realistic ENSO model called the Zebiak–Cane model. It's supposed to be the simplest model that exhibits fairly realistic ENSO behavior. I included some references. But I don't know what it looks like yet!`

You get this sort of behaviour for predator-prey relationships in simple models of ecosystems. My book says

`You get this sort of behaviour for predator-prey relationships in simple models of ecosystems. My book says > It has been found that reasonable predator-prey models may have interior, asymptotically stable equilibria, limit cycles, or both simultaneously.`

Yes, Hopf bifurcations are one of those universal phenomena... one of those things that make the world go round!

Somehow I'm more excited by all these dynamical systems ideas now that I'm interested in the climate, than back when I was learning some dynamical system theory just as abstract theoretical math/physics. Maybe it's because the climate is so complicated and hard to understand — so I'll eagerly grasp at any pattern that seems comprehensible!

`Yes, Hopf bifurcations are one of those universal phenomena... one of those things that make the world go round! Somehow I'm more excited by all these dynamical systems ideas now that I'm interested in the climate, than back when I was learning some dynamical system theory just as abstract theoretical math/physics. Maybe it's because the climate is so complicated and hard to understand — so I'll eagerly grasp at any pattern that seems comprehensible!`

I did something which is sort of what you wanted in Quantitative ecology.

I can't imagine what the two variables would be in the ENSO case. The ones on the ENSO page are too anti-correlated. The atmosphere just seems to 'echo' the ocean.

`I did something which is sort of what you wanted in [[Quantitative ecology]]. I can't imagine what the two variables would be in the ENSO case. The ones on the [[ENSO]] page are too anti-correlated. The atmosphere just seems to 'echo' the ocean.`

Whenever you have something that oscillates back and forth, its position and velocity go around in circles, approximately — so those are candidates for your two variables.

But I need to read more about ENSO models to give your question the answer it deserves!

`Whenever you have something that oscillates back and forth, its position and velocity go around in circles, approximately — so those are candidates for your two variables. But I need to read more about ENSO models to give your question the answer it deserves!`

I added a link to the paper "El Niño and the Delayed Action Oscillator".

`I added a link to the paper ["El Niño and the Delayed Action Oscillator"](http://arxiv.org/abs/physics/0603083).`

John said:

Sure, just let me take a look at the model :-)

I think it is a very sensible division of work if you do all the thinking and I do a bit of typing!

But seriously: Learning to program is not an easy task, it's an industrial art. I know a lot of people in IT-consulting and management who cannot program (beyond simple COBOL routines or something like that) and are very happy that they do not have to keep up to date with the current programming languages, frameworks, tools, paradigms...

`John said: <blockquote> <p> I would love it if one of you programming experts could make these changes in the program Tim already wrote! If you teach me enough I could do it myself, but it might be more efficient if I find and explain nice toy climate models, and you guys write the programs. Does that make sense? </p> </blockquote> Sure, just let me take a look at the model :-) I think it is a very sensible division of work if you do all the thinking and I do a bit of typing! But seriously: Learning to program is not an easy task, it's an industrial art. I know a lot of people in IT-consulting and management who cannot program (beyond simple COBOL routines or something like that) and are very happy that they do not have to keep up to date with the current programming languages, frameworks, tools, paradigms...`

Does anyone know what EOF (empirical orthogonal functions) analysis is and how it is related to Fourier and wavelet analysis?

`Does anyone know what EOF (empirical orthogonal functions) analysis is and how it is related to Fourier and wavelet analysis?`

John said:

In this paper there are no concrete model equations, only some general statements beginning with equation 3.1, which I don't quite understand...maybe the equations you mean are in one of the other papers?

I'm confused by the nomenclature of Kleeman, what he calls "propagator" is clearly not the Fokker-Planck operator, but only the deterministic part of a discretization of some kind. And I don't understand how he derives his equations for the mean and variance of the solution process.

But anyway, generalizing from one-dimensional to multi-dimensional equations with correlated white noise is not complicated, it is quite easy! So I'd agree, somewhat prematurely, with the statement

`John said: >First I talk about some equations that describe a Hopf bifurcation; then I talk about adding white noise to these equations. All this is discussed in this paper, which is free online: > * Richard Kleeman, Stochastic theories for the irregularity of the ENSO, in Stochastic Physics and Climate Modelling, eds. Tim Palmer and Paul Williams, Cambridge U. Press, Cambridge, 2010. In this paper there are no concrete model equations, only some general statements beginning with equation 3.1, which I don't quite understand...maybe the equations you mean are in one of the other papers? I'm confused by the nomenclature of Kleeman, what he calls "propagator" is clearly not the Fokker-Planck operator, but only the deterministic part of a discretization of some kind. And I don't understand how he derives his equations for the mean and variance of the solution process. But anyway, generalizing from one-dimensional to multi-dimensional equations with correlated white noise is not complicated, it is quite easy! So I'd agree, somewhat prematurely, with the statement <blockquote> <p> It seems to me that fairly small changes in the program we use to illustrate stochastic resonance could yield a program that would illustrate the Hopf-bifurcation-with-noise-added! </p> </blockquote>`

I think the ENSO is like a violin! When the bow is drawn across a string, it pulls it for a while, and then the string slips back a bit, only to be caught by the bow again. So the bow is like the trade winds, and the greater friction between bow and string when they are moving together is like the positive feedback between the warm water piling up in the West and the winds. When the warm water slips back towards the East, you get an El Nino!

The time scales are different (ms vs years) and so are the distances (mm vs 10,000km) - in both cases the ratio is about $10^10$. The bow against string makes a characteristic 'sawtooth' vibration: can you see this shape in the graphs of ocean temperature and SOI? ;-)

`I think the ENSO is like a violin! When the bow is drawn across a string, it pulls it for a while, and then the string slips back a bit, only to be caught by the bow again. So the bow is like the trade winds, and the greater friction between bow and string when they are moving together is like the positive feedback between the warm water piling up in the West and the winds. When the warm water slips back towards the East, you get an El Nino! The time scales are different (ms vs years) and so are the distances (mm vs 10,000km) - in both cases the ratio is about $10^10$. The bow against string makes a characteristic 'sawtooth' vibration: can you see this shape in the graphs of ocean temperature and SOI? ;-)`

EOF analysis is basically just principal components analysis (PCA), although a statistician told me once that there is some slight difference.

`EOF analysis is basically just principal components analysis (PCA), although a statistician told me once that there is some slight difference.`

Okay, so EOF is basically a different name for a discrete Karhunen–Loève expansion or Karhunen–Loève decomposition. Maybe the difference is that climate scientists don't care if the expansion is really an expansion into independent random variables?

`Okay, so EOF is basically a different name for a discrete Karhunen–Loève expansion or Karhunen–Loève decomposition. Maybe the difference is that climate scientists don't care if the expansion is really an expansion into independent random variables?`

I don't remember what the difference is. It may have something to do with spatial vs. spacetime decompositions. Neither EOFs nor PCA are decompositions into independent variables. They're just decompositions into uncorrelated variables.

`I don't remember what the difference is. It may have something to do with spatial vs. spacetime decompositions. Neither EOFs nor PCA are decompositions into independent variables. They're just decompositions into uncorrelated variables.`

Nathan said:

Oh, a classic blunder: I said independent when it should have been uncorrelated, thank God that no statistican caught me doing that...(I'd never hear the end of it :-)

Several authors seem to state that EOF and PCA are exactly the same, I haven't found one yet that defines them differently. Once I know a little bit more about it, I'll start a page about it on Azimuth...EOF seems to be a good keyword to find papers by climate scientists that tackle irregular data, too.

`Nathan said: <blockquote> <p> Neither EOFs nor PCA are decompositions into independent variables. They're just decompositions into uncorrelated variables. </p> </blockquote> Oh, a classic blunder: I said independent when it should have been uncorrelated, thank God that no statistican caught me doing that...(I'd never hear the end of it :-) Several authors seem to state that EOF and PCA are exactly the same, I haven't found one yet that defines them differently. Once I know a little bit more about it, I'll start a page about it on Azimuth...EOF seems to be a good keyword to find papers by climate scientists that tackle irregular data, too.`

Tim wrote:

Great! The model I'm thinking about right now is

extremelytoy-like. It's not a serious model of the ENSO. It's just the Hopf bifurcation plus noise. The equations are nicely visible near the bottom of our ENSO page... at least when the Azimuth Project wiki is working.Sorry, you're right! I meant this paper:

Stochastic Physics and Climate Modelling, eds. Tim Palmer and Paul Williams, Cambridge U. Press, Cambridge, 2010, pp. 287-306.If the Azimuth Project is down, you can see the equations here: they're equations 1.1, if you don't include noise, or equations 1.6, if you do. (We want to include noise!)

Figure 1 shows how the Hopf bifurcation is smoothed out by the presence of noise. But the kind of figure I'd like is much simpler: just a plot of a typical solution trajectory in the plane. Without noise, we should get pretty curves like in Figure 2 here. As we turn on noise, we should get various slightly interesting phenomena... nothing too amazing, but some "cycles with irregularity" which could be useful in climate modelling. There's a pretty good description in the above paper.

Yikes - I've got to run to a dinner with some of Lisa's friends now! I can answer more questions later, though.

`Tim wrote: > Sure, just let me take a look at the model :-) Great! The model I'm thinking about right now is _extremely_ toy-like. It's not a serious model of the ENSO. It's just the Hopf bifurcation plus noise. The equations are nicely visible near the bottom of our [[ENSO]] page... at least when the Azimuth Project wiki is working. > In [Kleiman's] paper there are no concrete model equations, only some general statements beginning with equation 3.1, which I don't quite understand...maybe the equations you mean are in one of the other papers? Sorry, you're right! I meant this paper: * Leela M. Frankcombe, Henk A. Dijkstra and Anna S. von Der Heyt, [The Atlantic Multidecadal Oscillation: a stochastic dynamical systems view](http://igitur-archive.library.uu.nl/phys/2009-0223-201312/rsta_2008_366_1875_2545_rsta20080031_web.pdf), in _Stochastic Physics and Climate Modelling_, eds. Tim Palmer and Paul Williams, Cambridge U. Press, Cambridge, 2010, pp. 287-306. If the Azimuth Project is down, you can see the equations here: they're equations 1.1, if you don't include noise, or equations 1.6, if you do. (We want to include noise!) Figure 1 shows how the Hopf bifurcation is smoothed out by the presence of noise. But the kind of figure I'd like is much simpler: just a plot of a typical solution trajectory in the plane. Without noise, we should get pretty curves like in Figure 2 [here](http://www.scholarpedia.org/article/Andronov-Hopf_bifurcation). As we turn on noise, we should get various slightly interesting phenomena... nothing too amazing, but some "cycles with irregularity" which could be useful in climate modelling. There's a pretty good description in the above paper. Yikes - I've got to run to a dinner with some of Lisa's friends now! I can answer more questions later, though.`

Actually, Graham Jones' graphic of "predator prey with noise" looks very similar to what I was trying to get Tim van Beek to do above! In particular, he shows how noise converts a stable fixed point into an irregular sort of cycle.

In the next issue of This Week's Finds, I'd love graphics illustrating a Hopf bifurcation with noise, to continue the explanation that can currently be found at the bottom of ENSO.

Graham is indeed studying a Hopf bifurcation with noise. But I want the mathematically most beautiful example:

$$ \frac{d x}{d t} = - \omega y + \beta x - x (x^2 + y^2) + \gamma noise $$ $$ \frac{d y}{d t} = - \omega x + \beta y - y (x^2 + y^2) + \gamma noise $$ Here $+ \gamma noise$ is my stupid way of saying that we're adding independent identically distributed white noise terms to each equation.

I'd love to see 2d trajectories just like Graham did for his example. To explain the idea, think we need to see them for $\beta = -1, 0, 1$ for a few values of $\gamma$ including $\gamma = 0$. I think we can just take $\omega = 1$ without loss of generality.

At least in the case $\gamma = 0$, I think we need to see 3 different trajectories for each value of $\beta$: we want pretty pictures just like this.

It might be fun to see trajectories with the same initial data but with noise included.

`Actually, Graham Jones' graphic of "predator prey with noise" looks very similar to what I was trying to get Tim van Beek to do above! In particular, he shows how noise converts a stable fixed point into an irregular sort of cycle. In the next issue of This Week's Finds, I'd love graphics illustrating a Hopf bifurcation with noise, to continue the explanation that can currently be found at the bottom of [[ENSO]]. Graham is indeed studying a Hopf bifurcation with noise. But I want the mathematically most beautiful example: $$ \frac{d x}{d t} = - \omega y + \beta x - x (x^2 + y^2) + \gamma noise $$ $$ \frac{d y}{d t} = - \omega x + \beta y - y (x^2 + y^2) + \gamma noise $$ Here $+ \gamma noise$ is my stupid way of saying that we're adding independent identically distributed white noise terms to each equation. I'd love to see 2d trajectories just like Graham did for his example. To explain the idea, think we need to see them for $\beta = -1, 0, 1$ for a few values of $\gamma$ including $\gamma = 0$. I think we can just take $\omega = 1$ without loss of generality. At least in the case $\gamma = 0$, I think we need to see 3 different trajectories for each value of $\beta$: we want pretty pictures just like [this](http://www.scholarpedia.org/wiki/images/thumb/2/2d/SubHopf.gif/450px-SubHopf.gif). It might be fun to see trajectories with the same initial data but with noise included.`

In This Week's Finds, I could also broaden out the discussion to include predator-prey models and include the graphics Graham has already made. That would make a nice example of how the same math shows up in different applications.

`In This Week's Finds, I could also broaden out the discussion to include predator-prey models and include the graphics Graham has already made. That would make a nice example of how the same math shows up in different applications.`

Graham could try it in R and I will try to do it in Java.

Whenever we specify multidimensional equations with noise, it is important to point out how the noise terms are correlated. When I read something like this (that's in one of the papers about the ENSO we have been discussing): $$ DX_t = ...+ \lambda d W_t $$ $$ DY_t = ...+ \lambda d W_t $$ ...I wonder if the author intends to say that the noise terms are identical or different. It is more precise this way: Let $(W_1, W_2)$ be two-dimensional white noise and $$ DX_t = ...+ \lambda_1 d W_{1, t} $$ $$ DY_t = ...+ \lambda_2 d W_{2, t} $$ This means that x and y are influenced by independent white noise processes.

Now to the textbook example of a Hopf bifurcation: I think that the SDE with $\beta \gt 0$ and independent noise processes are stochastically stiff near the limit circle, although the deterministic system is not stiff. This would mean that one should try a strong, implicit scheme like the implicit Milstein scheme. That should not be too difficult since the noise matrix is diagonal...

`Graham could try it in R and I will try to do it in Java. Whenever we specify multidimensional equations with noise, it is important to point out how the noise terms are correlated. When I read something like this (that's in one of the papers about the ENSO we have been discussing): $$ DX_t = ...+ \lambda d W_t $$ $$ DY_t = ...+ \lambda d W_t $$ ...I wonder if the author intends to say that the noise terms are identical or different. It is more precise this way: Let $(W_1, W_2)$ be two-dimensional white noise and $$ DX_t = ...+ \lambda_1 d W_{1, t} $$ $$ DY_t = ...+ \lambda_2 d W_{2, t} $$ This means that x and y are influenced by independent white noise processes. Now to the textbook example of a Hopf bifurcation: I think that the SDE with $\beta \gt 0$ and independent noise processes are stochastically stiff near the limit circle, although the deterministic system is not stiff. This would mean that one should try a strong, implicit scheme like the implicit Milstein scheme. That should not be too difficult since the noise matrix is diagonal...`

Either one would be fine with me. I'd love to get some pictures by the middle of next week. On Saturday the 25th I'll be leaving for Vietnam until January 5th. It would be nice to get a

This Week's Findsdone before then.I noticed that ambiguity when I wrote my last comment on this thread. I realized that I should give the different noise terms different names, but I was feeling lazy, so I explained it in words. If someone doesn't even explain in words which option they mean, they are being very, very naughty.

Of course I want my $\lambda_1$ and $\lambda_2$ to be equal since I want to preserve the circular symmetry of the problem, just to keep things pretty.

Wow, that sounds so fancy! I have no idea what "stiff" means here, much less "stochastically stiff". "Strong, implicit schemes" are also a mystery to me, as is the "implicit Milstein scheme". I'll have to look up these terms.

I thought you would just use some Euler-like step-by-step updating of $x(t)$ and $y(t)$ in the obvious way, giving each one an independent Gaussian "kick" along with the deterministic change. At least, that's what

I'ddo if nobody was payingme.Let me know if you can produce some pictures by, say, next Wednesday. If not, I'll just relax a bit.

If and when you do create some pictures, please tell me a bit about your algorithm! It would be nice if I could 1) explain the math, 2) include pictures, 3) say that these were produced as part of the Azimuth Code Project by Tim van Beek, and 4) say a tiny bit about the programming, to get programmers interested.

E.g.: I can look up stuff like "implicit Milstein scheme" and say stuff like "Tim decided to use an implicit Milstein scheme because this system is stochastically stiff near the limit cycle..." - and then explain what the hell all that stuff actually means! And that would seem very cool, at least to me.

The same sort of offer holds for Graham, or anyone else. But we should probably coordinate our activities a little bit to be efficient. E.g., maybe I should borrow Graham's expertise for the predator-prey applications of Hopf bifurcations.

`> Graham could try it in R and I will try to do it in Java. Either one would be fine with me. I'd love to get some pictures by the middle of next week. On Saturday the 25th I'll be leaving for Vietnam until January 5th. It would be nice to get a _This Week's Finds_ done before then. > ...I wonder if the author intends to say that the noise terms are identical or different. I noticed that ambiguity when I wrote my last comment on this thread. I realized that I should give the different noise terms different names, but I was feeling lazy, so I explained it in words. If someone doesn't even explain in words which option they mean, they are being very, very naughty. <img src = "http://math.ucr.edu/home/baez/emoticons/shame_on_you.gif" alt = ""/> Of course I want my $\lambda_1$ and $\lambda_2$ to be equal since I want to preserve the circular symmetry of the problem, just to keep things pretty. > Now to the textbook example of a Hopf bifurcation: I think that the SDE with $\beta \gt 0$ and independent noise processes are stochastically stiff near the limit circle, although the deterministic system is not stiff. This would mean that one should try a strong, implicit scheme like the implicit Milstein scheme. Wow, that sounds so fancy! I have no idea what "stiff" means here, much less "stochastically stiff". "Strong, implicit schemes" are also a mystery to me, as is the "implicit Milstein scheme". I'll have to look up these terms. I thought you would just use some Euler-like step-by-step updating of $x(t)$ and $y(t)$ in the obvious way, giving each one an independent Gaussian "kick" along with the deterministic change. At least, that's what _I'd_ do if nobody was paying _me_. Let me know if you can produce some pictures by, say, next Wednesday. If not, I'll just relax a bit. If and when you do create some pictures, please tell me a bit about your algorithm! It would be nice if I could 1) explain the math, 2) include pictures, 3) say that these were produced as part of the Azimuth Code Project by Tim van Beek, and 4) say a tiny bit about the programming, to get programmers interested. E.g.: I can look up stuff like "implicit Milstein scheme" and say stuff like "Tim decided to use an implicit Milstein scheme because this system is stochastically stiff near the limit cycle..." - and then explain what the hell all that stuff actually means! And that would seem very cool, at least to me. The same sort of offer holds for Graham, or anyone else. But we should probably coordinate our activities a little bit to be efficient. E.g., maybe I should borrow Graham's expertise for the predator-prey applications of Hopf bifurcations.`

No problem, I can explain all of this on the Azimuth project, starting with Stiff differential equation.

`No problem, I can explain all of this on the Azimuth project, starting with [[Stiff differential equation]].`

An explicit schema uses information of the current position only, example:

An implicit schema uses information about intermediate values, too, for example:

The implicit Milstein scheme is a certain algorithm for the numerical approximation of stochastic differential equation that is of the strong order 1.0.

Strong and weak approximations and orders are explained on stochastic differential equation (paragraph "Orders of Convergence").

`An explicit schema uses information of the current position only, example: * evaluate the first 5 terms of the Taylor series of a function f at x to determine f(x + h). An implicit schema uses information about intermediate values, too, for example: * evaluate the first 5 terms of the Taylor series of a function f at x to determine f(x + h/2), then evaluate the first 3 terms of f at the point f(x + h/2) (the approximate point you just calculated) to determine f(x + h). The implicit Milstein scheme is a certain algorithm for the numerical approximation of stochastic differential equation that is of the strong order 1.0. Strong and weak approximations and orders are explained on [[stochastic differential equation]] (paragraph "Orders of Convergence").`

Thanks, Tim! It'll take me a while to absorb this information, but it's good stuff.

A while back, Nathan wrote:

I'm really enjoying this paper, probably because the model it describes is "ideally suited for student projects both at high school and undergraduate level." I like how it uses a delay-differential equation

$$ \frac{d}{d t} T(t) = k T(t) - b T(t)^3 - A T(t - \Delta)^3 $$ ($k, b, A, \Delta \gt 0$) to model the possible effect of oceanic Rossby waves:

I need to find out what Kelvin waves are! Anyway:

I'm not sure I fully understand it, but it's an elaboration of what Kessler was talking near the end of week307, and it would be

incredibly groovyif it were true: slow waves in the thermocline bouncing across the ocean and interacting with the weather, causing complicated oscillations! And the model seems to fit the data really well!`Thanks, Tim! It'll take me a while to absorb this information, but it's good stuff. A while back, Nathan wrote: >I added a link to the paper ["El Niño and the Delayed Action Oscillator"](http://arxiv.org/abs/physics/0603083). I'm really enjoying this paper, probably because the model it describes is "ideally suited for student projects both at high school and undergraduate level." I like how it uses a delay-differential equation $$ \frac{d}{d t} T(t) = k T(t) - b T(t)^3 - A T(t - \Delta)^3 $$ ($k, b, A, \Delta \gt 0$) to model the possible effect of [[oceanic Rossby waves]]: > Lastly, the model also considers equatorially-trapped ocean waves propagating across the Pacific, before interacting back with the central Pacific region after a certain time delay. These ocean waves are “hidden” Rossby waves which move westward on the [[thermocline]], reflect off the rigid continental boundary in the West and then return eastward along the equator as Kelvin waves. I need to find out what [Kelvin waves](http://en.wikipedia.org/wiki/Kelvin_wave#Equatorial_Kelvin_wave) are! Anyway: > The [...] delay term has a negative coefficient representing a negative feedback. To see the reason for this, let us consider a warm SST [sea surface temperature] perturbation in the coupled region. This produces a westerly wind response that deepens the thermocline locally (immediate positive feedback), but at the same time, the wind perturbations produce divergent westward propagating Rossby waves that return after time $\Delta$ to create upwelling and cooling, reducing the original perturbation. I'm not sure I fully understand it, but it's an elaboration of what Kessler was talking near the end of [week307](http://math.ucr.edu/home/baez/week307.html), and it would be _incredibly groovy_ if it were true: slow waves in the thermocline bouncing across the ocean and interacting with the weather, causing complicated oscillations! And the model seems to fit the data really well!`

I'll let Tim do the SDEs here, though maybe I could help with graphics. I've heard of 'stiff' in this context but not 'stochastically stiff '. My impression from the predator-prey example (with step-by-step updating of x(t) and y(t) in the obvious way) is that it is numerically awkward near the bifurcation without noise.

`I'll let Tim do the SDEs here, though maybe I could help with graphics. I've heard of 'stiff' in this context but not 'stochastically stiff '. My impression from the predator-prey example (with step-by-step updating of x(t) and y(t) in the obvious way) is that it is numerically awkward near the bifurcation without noise.`

There is a sign wrong in

$$ \frac{d x}{d t} = - \omega y + \beta x - x (x^2 + y^2) + \gamma noise $$ $$ \frac{d y}{d t} = - \omega x + \beta y - y (x^2 + y^2) + \gamma noise $$ One of the RHS's should start with a $+$. I've fixed the ENSO page.

`There is a sign wrong in $$ \frac{d x}{d t} = - \omega y + \beta x - x (x^2 + y^2) + \gamma noise $$ $$ \frac{d y}{d t} = - \omega x + \beta y - y (x^2 + y^2) + \gamma noise $$ One of the RHS's should start with a $+$. I've fixed the [[ENSO]] page.`

Thanks, Graham. I changed your choice of where the minus sign went, to make the equation consistent with the equation below, in polar coordinates.

`Thanks, Graham. I changed your choice of where the minus sign went, to make the equation consistent with the equation below, in polar coordinates.`

Since I'd like to transform the equations in the presence of noise from cartesian coordinates to polar coordinates, I created the page Itô formula to do it there. Too bad that I did not find it, I'm almost sure I'm not the first one who has to do that! But we can use this opportunity to get a little bit more interesting mathematics on the Wiki.

`Since I'd like to transform the equations in the presence of noise from cartesian coordinates to polar coordinates, I created the page [[Itô formula]] to do it there. Too bad that I did not find it, I'm almost sure I'm not the first one who has to do that! But we can use this opportunity to get a little bit more interesting mathematics on the Wiki.`

I implemented the textbook Hopf bifurcation model with noise using the Euler scheme, I think I'll go for the delayed action oscillator next.

BTW: Peter Kloeden has published last month a simplified proof of the numerical stability of the Milstein scheme:

Milstein for stochastic delay equations.

So, this stuff gets us into top-notch math research :-) (Not what I'm doing, that is a textbook implementation of textbook models :-) but the background math material.)

`I implemented the textbook Hopf bifurcation model with noise using the Euler scheme, I think I'll go for the delayed action oscillator next. BTW: Peter Kloeden has published last month a simplified proof of the numerical stability of the Milstein scheme: <a href="http://www.maths.manchester.ac.uk/~shardlow/papers/taylor_delay.pdf">Milstein for stochastic delay equations</a>. So, this stuff gets us into top-notch math research :-) (Not what I'm doing, that is a textbook implementation of textbook models :-) but the background math material.)`

Tim wrote:

Great!!! (Tim and I have been corresponding via email about this.)

I'll be pestering you for pretty pictures. I didn't get very far writing up an explanation of the Hopf bifurcation with noise today, because I made the mistake of trying to understand more complicated ENSO models and how they lead to this simpler model. Whew! Complicated stuff!

If you're interested in the ENSO applications of the delayed action oscillator, the paper Nathan pointed us to is very good:

It describes a sequence of models of increasing complexity, all rather simple. Check out how well the simple one on our ENSO page fits the data! It would be very nice to get some graphs of this.

`Tim wrote: > I implemented the textbook Hopf bifurcation model with noise using the Euler scheme. Great!!! (Tim and I have been corresponding via email about this.) I'll be pestering you for pretty pictures. I didn't get very far writing up an explanation of the Hopf bifurcation with noise today, because I made the mistake of trying to understand more complicated ENSO models and how they lead to this simpler model. Whew! Complicated stuff! If you're interested in the ENSO applications of the delayed action oscillator, the paper Nathan pointed us to is very good: * Ian Boutle, Richard H. S. Taylor, and Rudolf A. Roemer, [El Nino and the delayed action oscillator](http://arxiv.org/abs/physics/0603083). It describes a sequence of models of increasing complexity, all rather simple. Check out how well the simple one on our [[ENSO]] page fits the data! It would be very nice to get some graphs of this.`

Since more and more people are starting to want El Niño related data, I've started a section here:

If you find a good source of El Niño related data, or create your own database, please add a link here.

Interestingly, I found a bunch of data on the U. C. Irvine Machine Learning Repository. I was trying to find what had been done on using machine learning to predict El Niños. I didn't find an answer there, but I found a bunch of ocean temperature data!

I'll quote this part of what I wrote:

TAO arrayThe

TAO array, renamed theTAO/TRITON arrayon 1 January 2000, consists of approximately 70 moorings in the Tropical Pacific Ocean, telemetering oceanographic and meteorological data to shore in real-time via the Argos satellite system.You can download data here:

There is an older data set at U.C. Irvine:

The data is stored in an ASCII files with one observation per line. Spaces separate fields and periods (.) denote missing values.

Among other things they write:

`Since more and more people are starting to want El Niño related data, I've started a section here: * [ENSO - data](http://www.azimuthproject.org/azimuth/show/ENSO#Data). If you find a good source of El Niño related data, or create your own database, please add a link here. Interestingly, I found a bunch of data on the U. C. Irvine Machine Learning Repository. I was trying to find what had been done on using machine learning to predict El Niños. I didn't find an answer there, but I found a bunch of ocean temperature data! I'll quote this part of what I wrote: <hr/> **TAO array** The **TAO array**, renamed the **[TAO/TRITON array](http://www.pmel.noaa.gov/tao/proj_over/proj_over.html)** on 1 January 2000, consists of approximately 70 moorings in the Tropical Pacific Ocean, telemetering oceanographic and meteorological data to shore in real-time via the Argos satellite system. <img src = "http://www.pmel.noaa.gov/tao/images/array.gif" alt = ""/> You can download data here: * [TAO data display and delivery](http://www.pmel.noaa.gov/tao/disdel/disdel.html). There is an older data set at U.C. Irvine: * [El Nino Data Set](https://archive.ics.uci.edu/ml/datasets/El+Nino), UCI Machine Learning Repository. The data is stored in an ASCII files with one observation per line. Spaces separate fields and periods (.) denote missing values. Among other things they write: > This data was collected with the Tropical Atmosphere Ocean (TAO) array which was developed by the international Tropical Ocean Global Atmosphere (TOGA) program. The TAO array consists of nearly 70 moored buoys spanning the equatorial Pacific, measuring oceanographic and surface meteorological variables critical for improved detection, understanding and prediction of seasonal-to-interannual climate variations originating in the tropics, most notably those related to the El Nino/Southern Oscillation (ENSO) cycles. > The moorings were developed by National Oceanic and Atmospheric Administration's (NOAA) Pacific Marine Environmental Laboratory (PMEL). Each mooring measures air temperature, relative humidity, surface winds, sea surface temperatures and subsurface temperatures down to a depth of 500 meters and a few a of the buoys measure currents, rainfall and solar radiation. The data from the array, and current updates, can be viewed on the web at the this address . > The data consists of the following variables: date, latitude, longitude, zonal winds (west<0, east>0), meridional winds (south<0, north>0), relative humidity, air temperature, sea surface temperature and subsurface temperatures down to a depth of 500 meters. Data taken from the buoys from as early as 1980 for some locations. Other data that was taken in various locations are rainfall, solar radiation, current levels, and subsurface temperatures. > **Variable Characteristics** > The latitude and longitude in the data showed that the bouys moved around to different locations. The latitude values stayed within a degree from the approximate location. Yet the longitude values were sometimes as far as five degrees off of the approximate location. > Looking at the wind data, both the zonal and meridional winds fluctuated between -10 m/s and 10 m/s. The plot of the two wind variables showed no linear relationship. Also, the plots of each wind variable against the other three meteorolgical data showed no linear relationships. > The relative humidity values in the tropical Pacific were typically between 70% and 90%. > Both the air temperature and the sea surface temperature fluctuated between 20 and 30 degrees Celsius. The plot of the two temperatures variables shows a positive linear relationship existing. The two temperatures when each plotted against time also have similar plot designs. Plots of the other meteorological variables against the temperature variables showed no linear relationship. > There are missing values in the data. As mentioned earlier, not all buoys are able to measure currents, rainfall, and solar radiation, so these values are missing dependent on the individual buoy. The amount of data available is also dependent on the buoy, as certain buoys were commissioned earlier than others. > All readings were taken at the same time of day.`

John be careful about these ready-to-go databases for machine learning from different places, often they interpolated the data or altered it due to missing data or other considerations.

I like to investigate the originals e.g. from NASA or NOAA or some other multi-national repository before using the data seriously.

Dara

`>Interestingly, I found a bunch of data on the U. C. Irvine Machine Learning Repository. John be careful about these ready-to-go databases for machine learning from different places, often they interpolated the data or altered it due to missing data or other considerations. I like to investigate the originals e.g. from NASA or NOAA or some other multi-national repository before using the data seriously. Dara`

Dara - I updated the wiki page and also my comment before yours, adding a link to the original data. Thanks for the warning. I will follow your advice.

However, I believe the database at U.C. Irvine does not try to "fill in" missing data, since they say:

`Dara - I updated the wiki page and also my comment before yours, adding a link to the original data. Thanks for the warning. I will follow your advice. However, I believe the database at U.C. Irvine does not try to "fill in" missing data, since they say: > There are missing values in the data.`

This site has nice regularly updated animations of sea surface temperature anomalies. You can watch the potential El Niño forming now:

This page has El Niño predictions:

As of 10 July 2014:

You can see the pictures I was too lazy to copy over at that page.

`[This site](http://www.ospo.noaa.gov/Products/ocean/sst/anomaly/anim_6mp.html) has nice regularly updated animations of sea surface temperature anomalies. You can watch the potential El Niño forming now: <img src = "http://www.ospo.noaa.gov/data/sst/anomaly/anomalyps_6m.gif" alt = ""/> [This page](http://www.cpc.ncep.noaa.gov/products/analysis_monitoring/enso_advisory/ensodisc.html) has El Niño predictions: As of 10 July 2014: > The chance of El Niño is about 70% during the Northern Hemisphere summer and is close to 80% during the fall and early winter. > During June 2014, above-average sea surface temperatures (SST) were most prominent in the eastern equatorial Pacific, with weakening evident near the International Date Line (Fig. 1). <img src = "http://www.cpc.ncep.noaa.gov/products/analysis_monitoring/enso_advisory/figure1.gif" alt = ""/> > This weakening was reflected in a decrease to +0.3oC in the Niño-4 index (Fig. 2). <img src = "http://www.cpc.ncep.noaa.gov/products/analysis_monitoring/enso_advisory/figure2.gif" alt = ""/> > The Niño-3.4 index remained around +0.5°C throughout the month, while the easternmost Niño-3 and Niño-1+2 indices are +1.0°C or greater. Subsurface heat content anomalies (averaged between 180o-100oW) have decreased substantially since late March 2014 and are now near average (Fig. 3). <img src = "http://www.cpc.ncep.noaa.gov/products/analysis_monitoring/enso_advisory/figure3.gif" alt = ""/> > However, above-average subsurface temperatures remain prevalent near the surface (down to 100m depth) in the eastern half of the Pacific (Fig. 4). The upper-level and low-level winds over the tropical Pacific remained near average, except for low-level westerly anomalies over the eastern Pacific. Convection was enhanced near and just west of the Date Line and over portions of Indonesia (Fig. 5). Still, the lack of a clear and consistent atmospheric response to the positive SSTs indicates ENSO-neutral. > Over the last month, no significant change was evident in the model forecasts of ENSO, with the majority of models indicating El Niño onset within June-August and continuing into early 2015 (Fig. 6). The chance of a strong El Niño is not favored in any of the ensemble averages for Niño-3.4. At this time, the forecasters anticipate El Niño will peak at weak-to-moderate strength during the late fall and early winter (3-month values of the Niño-3.4 index between 0.5°C and 1.4°C). The chance of El Niño is about 70% during the Northern Hemisphere summer and is close to 80% during the fall and early winter (click CPC/IRI consensus forecast for the chance of each outcome). You can see the pictures I was too lazy to copy over at that page.`

OK yes John thnaks - you spoilt my morning. These images are unfortunately too good for getting a glimpse about what's going on at El Nino. I spent 2 hours blocking my browser with over 50 tabs simultainously being open each containing the global images of monthly anomalies of the years 2006, 2007, 2009, 2010, 2012, 2013, and half of 2014 and jumping forth an back between the tabs. It was so exiting that I had to appease me with three fat butterbreads. I got a rather different image about El Nino than as what I had sofar understood from the explanations. In particular by looking at two supposedly El Nino's in 2006 and 2009 in comparision to the rest, it looks that the heat transfer of temperatures over the northern atlantic to the basin in front of northern south America seem to play a bigger role. That is it looks as if the transfered heat content which had been piled up in the northern atlantic is "bigger" than the heat content over the islands between Australia and China and partially over the indian ocean than this may at least contribute to an El Nino condition ("a chimney over South America", so to say). For saying wether this "leads" to an El Nino condition I haven't seen enough data to tell, one would need to look at way more El nino images and probably this aspect of atlantic heat transfer vs Asia is not enough. But I currently think that it probably should be taken into consideration within any model for prediction. But as said I just looked at it for two hours. Temperatures over the northern atlantic and south America are currently piling up, but there is still quite some heat content over Asia. So just by judging with respect to this heat content transfer it currently wouldn't be so clear wether there would be an El Nino in autumn. This would depend how temperatures develop in the next months. Furthermore that images are also not so good for detecting differences in heat content.

`>You can watch the potential El Niño forming now: OK yes John thnaks - you spoilt my morning. These images are unfortunately too good for getting a glimpse about what's going on at El Nino. I spent 2 hours blocking my browser with over 50 tabs simultainously being open each containing the global images of monthly anomalies of the years 2006, 2007, 2009, 2010, 2012, 2013, and half of 2014 and jumping forth an back between the tabs. It was so exiting that I had to appease me with three fat butterbreads. I got a rather different image about El Nino than as what I had sofar understood from the explanations. In particular by looking at two supposedly El Nino's in 2006 and 2009 in comparision to the rest, it looks that the heat transfer of temperatures over the northern atlantic to the basin in front of northern south America seem to play a bigger role. That is it looks as if the transfered heat content which had been piled up in the northern atlantic is "bigger" than the heat content over the islands between Australia and China and partially over the indian ocean than this may at least contribute to an El Nino condition ("a chimney over South America", so to say). For saying wether this "leads" to an El Nino condition I haven't seen enough data to tell, one would need to look at way more El nino images and probably this aspect of atlantic heat transfer vs Asia is not enough. But I currently think that it probably should be taken into consideration within any model for prediction. But as said I just looked at it for two hours. Temperatures over the northern atlantic and south America are currently piling up, but there is still quite some heat content over Asia. So just by judging with respect to this heat content transfer it currently wouldn't be so clear wether there would be an El Nino in autumn. This would depend how temperatures develop in the next months. Furthermore that images are also not so good for detecting differences in heat content.`

Hi Nad - for some reason I'm just noticing your comment now! It sounds interesting; we need to stare at El Niños, eat butterbreads and form some hypotheses.

People are currently expecting a rather weak El Niño this autumn.

I came to this thread merely because I added some links to

namely data for air pressures at Darwin and Tahiti.

`Hi Nad - for some reason I'm just noticing your comment now! It sounds interesting; we need to stare at El Niños, eat butterbreads and form some hypotheses. People are currently expecting a rather weak El Niño this autumn. I came to this thread merely because I added some links to * [ENSO - Southern Oscillation Index](http://www.azimuthproject.org/azimuth/show/ENSO#SOI). namely data for air pressures at Darwin and Tahiti.`

yes it currently looks as if there is not so much heat over east asia anymore, but still quite an amount in front of south america and north america:

http://www.ospo.noaa.gov/data/sst/anomaly/2014/anomnight.7.28.2014.gif

this points to an El Nino, but it doesn't look yet as the one in 1997:

http://www.ospo.noaa.gov/data/sst/anomaly/1997/anomnight.7.1.1997.gif

I just try to understand this El Nino better (and actually there were also other things I was looking for). Do I need to do this? I don't know. It's currently more of a gut feeling that understanding El Nino better might be good....but it is probably not what my environment thinks I need and should do and it doesn't look as if it is what I should do for having some income at old age. And sitting in front of the screen and eating butter breads is not optimal for my health either.

I was also looking at the temperatures of the north and southern hemissphere. I am not fully sure however wether the anomalies are not spacially weighted. If not it looks that the summer SST in the northern hemissphere are hotter on average than in the southern hemissphere. May be you remember that I have this prevailing suspicion that there are differences in the solar irradiation of northern and southern hemisphere.

Another thing that looked quite disconcerting is that already in march (the other years show the same) there is a hot spot at a place which looks as being around the Barents Sea: http://www.ospo.noaa.gov/data/sst/anomaly/2014/anomnight.3.3.2014.gif

It was even there in 1997: http://www.ospo.noaa.gov/data/sst/anomaly/1997/anomnight.3.4.1997.gif but now it looks as if it extends behind the Svalbard.

This warming is probably due to clathrates ??

`yes it currently looks as if there is not so much heat over east asia anymore, but still quite an amount in front of south america and north america: <a href = "http://www.ospo.noaa.gov/data/sst/anomaly/2014/anomnight.7.28.2014.gif">http://www.ospo.noaa.gov/data/sst/anomaly/2014/anomnight.7.28.2014.gif</a> <img src = "http://www.ospo.noaa.gov/data/sst/anomaly/2014/anomnight.7.28.2014.gif" alt = ""/> this points to an El Nino, but it doesn't look yet as the one in 1997: <a href="http://www.ospo.noaa.gov/data/sst/anomaly/1997/anomnight.7.1.1997.gif">http://www.ospo.noaa.gov/data/sst/anomaly/1997/anomnight.7.1.1997.gif</a> <img src = "http://www.ospo.noaa.gov/data/sst/anomaly/1997/anomnight.7.1.1997.gif" alt = ""/> >we need to stare at El Niños, eat butterbreads and form some hypotheses. I just try to understand this El Nino better (and actually there were also other things I was looking for). Do I need to do this? I don't know. It's currently more of a gut feeling that understanding El Nino better might be good....but it is probably not what my environment thinks I need and should do and it doesn't look as if it is what I should do for having some income at old age. And sitting in front of the screen and eating butter breads is not optimal for my health either. I was also looking at the temperatures of the north and southern hemissphere. I am not fully sure however wether the anomalies are not spacially weighted. If not it looks that the summer SST in the northern hemissphere are hotter on average than in the southern hemissphere. May be you remember that I have this prevailing suspicion that there are differences in the solar irradiation of northern and southern hemisphere. Another thing that looked quite disconcerting is that already in march (the other years show the same) there is a hot spot at a place which looks as being around the Barents Sea: <a href="http://www.ospo.noaa.gov/data/sst/anomaly/2014/anomnight.3.3.2014.gif">http://www.ospo.noaa.gov/data/sst/anomaly/2014/anomnight.3.3.2014.gif</a> It was even there in 1997: <a href="http://www.ospo.noaa.gov/data/sst/anomaly/1997/anomnight.3.4.1997.gif">http://www.ospo.noaa.gov/data/sst/anomaly/1997/anomnight.3.4.1997.gif</a> but now it looks as if it extends behind the Svalbard. This warming is probably due to clathrates ??`