It looks like you're new here. If you want to get involved, click one of these buttons!

- All Categories 2.4K
- Chat 505
- Study Groups 21
- Petri Nets 9
- Epidemiology 4
- Leaf Modeling 2
- Review Sections 9
- MIT 2020: Programming with Categories 51
- MIT 2020: Lectures 20
- MIT 2020: Exercises 25
- Baez ACT 2019: Online Course 339
- Baez ACT 2019: Lectures 79
- Baez ACT 2019: Exercises 149
- Baez ACT 2019: Chat 50
- UCR ACT Seminar 4
- General 75
- Azimuth Code Project 111
- Statistical methods 4
- Drafts 10
- Math Syntax Demos 15
- Wiki - Latest Changes 3
- Strategy 113
- Azimuth Project 1.1K
- - Spam 1
- News and Information 148
- Azimuth Blog 149
- - Conventions and Policies 21
- - Questions 43
- Azimuth Wiki 719

## Comments

Sure, that explains it, even if I'm less enamoured of it. We want Bayes linear regression, perhaps with these same predictor terms, but Bayes. For instance MCMCregress in the R package MCMCpack. If not Gaussian responses, there are alternatives in the same package.

`Sure, that explains it, even if I'm less enamoured of it. We want Bayes linear regression, perhaps with these same predictor terms, but Bayes. For instance MCMCregress in the R package MCMCpack. If not Gaussian responses, there are alternatives in the same package.`

This is nifty applying the regression to the Sydney tidal gauge data. From 1956 to 2008, there is an extremely strong biennial (2-year) Cosine period to the sea-level height anomaly (SLHA). Top panel, in yellow with amplitude of the Cosine term in the yellow box.

However, from 1900 to 1955, the phase of the biennial Cosine is flipped by 180 degrees. See bottom panel, amplitude with opposite sign.

An exact biennial oscillation is symmetric with respect to starting on an even versus odd year. How does it decide to start on an even or odd? The weaker 1/2 year period Cosine term also flips from one interval to the next.

North America Salmon populations know about this even vs odd as well

https://forum.azimuthproject.org/discussion/comment/13571/#Comment_13571

`This is nifty applying the regression to the Sydney tidal gauge data. From 1956 to 2008, there is an extremely strong biennial (2-year) Cosine period to the sea-level height anomaly (SLHA). Top panel, in yellow with amplitude of the Cosine term in the yellow box. However, from 1900 to 1955, the phase of the biennial Cosine is flipped by 180 degrees. See bottom panel, amplitude with opposite sign. ![tides](http://imageshack.com/a/img905/5449/AVPDVp.png) An exact biennial oscillation is symmetric with respect to starting on an even versus odd year. How does it decide to start on an even or odd? The weaker 1/2 year period Cosine term also flips from one interval to the next. North America Salmon populations know about this even vs odd as well https://forum.azimuthproject.org/discussion/comment/13571/#Comment_13571 ![salmon](http://imageshack.com/a/img673/4654/oCD718.gif)`

The recent finding that the earth's gravitational pull can influence the moon is making the rounds now:

http://www.sciencedaily.com/releases/2015/09/150915162512.htm

Applying similar reasoning to the QBO should be well accepted by comparison. A mass that can easily move due to gravitational effects is the one that has the least physical constraints. The ocean is one of these and the evidence is tidal action. Deeper within the ocean, the slight density difference across the thermocline is even more compliant and so the lunar pull here results in the

seeminglyerratic oscillations of ENSO. And consider that the QBO takes place in the stratosphere, where the density of the atmosphere is so small that a lunar pull can bulge it very easily. This has ramifications for forcing the stratospheric winds in certain directions -- which is manifested by the lunar Draconic and Synodic periodicities we see in the QBO time series. And way at the top of the stratosphere where the density is extremely reduced, the the bi-annual (twice a year) solar hemispheric see-saw takes precedent.Something has to give here because the scientific establishment can't pick and choose as to what phenomena can demonstrate lunar forcing and which ones won't.

Regarding the fundamental behavior behind the QBO, one of the biggest impediments to our understanding is based on what Richard Lindzen had written over 40 years ago. Since that time, Lindzen has become an anti-AGW zealot who disparages other climate scientists as evidenced here, the Lindzen paradox.

It is now clear that the QBO is primarily driven by the same lunar forces that govern the tides, but that Lindzen's original model has nothing to say about lunar forcing.

We shouldn't be concerned that Lindzen's theory needs to be revised. It's the data that has the last word.

`The recent finding that the earth's gravitational pull can influence the moon is making the rounds now: http://www.sciencedaily.com/releases/2015/09/150915162512.htm Applying similar reasoning to the QBO should be well accepted by comparison. A mass that can easily move due to gravitational effects is the one that has the least physical constraints. The ocean is one of these and the evidence is tidal action. Deeper within the ocean, the slight density difference across the thermocline is even more compliant and so the lunar pull here results in the *seemingly* erratic oscillations of ENSO. And consider that the QBO takes place in the stratosphere, where the density of the atmosphere is so small that a lunar pull can bulge it very easily. This has ramifications for forcing the stratospheric winds in certain directions -- which is manifested by the lunar Draconic and Synodic periodicities we see in the QBO time series. And way at the top of the stratosphere where the density is extremely reduced, the the bi-annual (twice a year) solar hemispheric see-saw takes precedent. Something has to give here because the scientific establishment can't pick and choose as to what phenomena can demonstrate lunar forcing and which ones won't. --- Regarding the fundamental behavior behind the QBO, one of the biggest impediments to our understanding is based on what Richard Lindzen had written over 40 years ago. Since that time, Lindzen has become an anti-AGW zealot who disparages other climate scientists as evidenced here, [the Lindzen paradox](http://davidappell.blogspot.com/2014/01/richard-lindzens-paradox-for-ages.html). It is now clear that the QBO is primarily driven by the same lunar forces that govern the tides, but that Lindzen's [original model](http://www-eaps.mit.edu/faculty/lindzen/qubieoscil.pdf) has nothing to say about lunar forcing. We shouldn't be concerned that Lindzen's theory needs to be revised. It's the data that has the last word.`

I was looking at some old posts that I have written and found this intriguing one on pulling out a 2-month periodicity in sea-level rise data:

http://theoilconundrum.blogspot.com/2013/04/filtering-sea-level-rise.html

I wrote :

This paper also finds a 50 to 66 day period in Kelvin waves I. Polo, A. Lazar, B. Rodriguez‐Fonseca, and S. Arnault, “Oceanic Kelvin waves and tropical Atlantic intraseasonal variability: 1. Kelvin wave characterization,” Journal of Geophysical Research: Oceans (1978–2012), vol. 113, no. C7, 2008. http://onlinelibrary.wiley.com/doi/10.1029/2007JC004495/full

Like the 2-year oscillation in tidal gauge SLHA, this 2-month cycle in the global mean is curious. Seasons follow a 1 year period (obviously) and lunar around 1 month. If these are both period doublings, what causes it?

`I was looking at some old posts that I have written and found this intriguing one on pulling out a 2-month periodicity in sea-level rise data: http://theoilconundrum.blogspot.com/2013/04/filtering-sea-level-rise.html ![2month](http://img546.imageshack.us/img546/3079/sealevelrise.gif) I wrote : > "Like tides, this phenomena is more similar to the sloshing of water in a bucket. (This particular two month cycle is not a tide as lunar tides such as neap and spring tides have monthly or biweekly periods)." This paper also finds a 50 to 66 day period in Kelvin waves I. Polo, A. Lazar, B. Rodriguez‐Fonseca, and S. Arnault, “Oceanic Kelvin waves and tropical Atlantic intraseasonal variability: 1. Kelvin wave characterization,” Journal of Geophysical Research: Oceans (1978–2012), vol. 113, no. C7, 2008. http://onlinelibrary.wiley.com/doi/10.1029/2007JC004495/full Like the 2-year oscillation in tidal gauge SLHA, this 2-month cycle in the global mean is curious. Seasons follow a 1 year period (obviously) and lunar around 1 month. If these are both period doublings, what causes it?`

I assume the reason for the Bayes is to get better sensitivity metrics out of the modeled data?

On the Entroplet served ENSO model, I included an Ornsetein-Uhlenbeck random walk generator so I can test for overfitting. Pick the O-U time series instead of the SOI time series, and you can see how well the model will fit red noise. If it fits it too well, then it is probably a case of too many factors in the model.

`>"Sure, that explains it, even if I'm less enamoured of it. We want Bayes linear regression, perhaps with these same predictor terms, but Bayes. For instance MCMCregress in the R package MCMCpack. If not Gaussian responses, there are alternatives in the same package." I assume the reason for the Bayes is to get better sensitivity metrics out of the modeled data? On the Entroplet served ENSO model, I included an Ornsetein-Uhlenbeck random walk generator so I can test for overfitting. Pick the O-U time series instead of the SOI time series, and you can see how well the model will fit red noise. If it fits it too well, then it is probably a case of too many factors in the model.`

James Hansen and colleagues have a fair bit to say about Southern Oceans and ENSO in a reply to critiques of their recent investigation. The "bit to say" is available.

`James Hansen and colleagues have a fair bit to say about Southern Oceans and ENSO in a reply to critiques of their recent investigation. The "bit to say" [is available](http://csas.ei.columbia.edu/2015/09/21/predictions-implicit-in-ice-melt-paper-and-global-implications/).`

The figure Hansen and colleagues show indicates a significant shift in the Southern Ocean starting after 1975 and peaking in 1981. It then returns before 2000. This is approximately the same interval where I have to invert the phase in the SOI model to achieve a good fit. (I am not sure why they have double vertical lines on 5-year intervals, as it makes it hard to guess the yearly alignment)

This was the model fit.

I don't know whether the two are related. The Southern Ocean is also known as the Antarctic Ocean and so is geophyscially separated from the equatorial Pacific in which the SOI is measured.

`The figure Hansen and colleagues show indicates a significant shift in the Southern Ocean starting after 1975 and peaking in 1981. It then returns before 2000. This is approximately the same interval where I have to invert the phase in the SOI model to achieve a good fit. (I am not sure why they have double vertical lines on 5-year intervals, as it makes it hard to guess the yearly alignment) ![so](http://csas.ei.columbia.edu/files/2015/09/Fig.-3.-Monthly-upper-chart-and-12-month-running-mean-lower-charts-of-Nino-3.4-temperature-index-Southern-Ocean-south-of-60%C2%B0S-sea-surface-temperature-anomaly-and-Southern-Ocean-sea-ice-area.-620x403.png) This was the model fit. ![model](http://imageshack.com/a/img537/6035/aF9wlD.png) I don't know whether the two are related. The Southern Ocean is also known as the Antarctic Ocean and so is geophyscially separated from the equatorial Pacific in which the SOI is measured.`

I dug up another old QBO machine learning run I did from several months ago. This is complementary to the analysis of

https://forum.azimuthproject.org/discussion/comment/14843/#Comment_14843

but is an independent run

The two factors have periods when unaliased that match the draconic and synodic/sidereal lunar month, with errors of 0.015%

The third value in the mix is a period when unaliased is a ~17-year modulated period of unfolded ~27.15 days, which may be solar cycle related

Mošna, Z., P. Šauli, and O. Santolık. "Analysis of Critical Frequencies in the Ionosphere." WDS’Proceedings of Contributed Papers (2008): 172-177.

This is turning into a rather precise analysis that could benefit from having an alternative model to compare against.

The larger argument hinges on the fact that the same frequencies are used to reconstructively model both the QBO and ENSO data sets. I don't know exactly how to quantify the likelihood of that being due to chance.

BTW, I am getting criticisms from weird corners of the science blogosphere that assert that I am stealing these QBO and ENSO ideas from them without giving proper attribution.

`I dug up another old QBO machine learning run I did from several months ago. This is complementary to the analysis of https://forum.azimuthproject.org/discussion/comment/14843/#Comment_14843 but is an independent run ![eureqa](http://imageshack.com/a/img912/5811/Z87ie7.jpg) ~~~~ aliased freq period equiv days actual % error 2.663161 2.359296 27.20828 27.21222=draconic -0.01449 2.300462 2.731272 27.32579 27.32166=sidereal 0.015093 ~~~~ The two factors have periods when unaliased that match the draconic and synodic/sidereal lunar month, with errors of 0.015% The third value in the mix is a period when unaliased is a ~17-year modulated period of unfolded ~27.15 days, which may be solar cycle related ![solar](http://imageshack.com/a/img905/2765/hsX5wM.gif) Mošna, Z., P. Šauli, and O. Santolık. "Analysis of Critical Frequencies in the Ionosphere." WDS’Proceedings of Contributed Papers (2008): 172-177. This is turning into a rather precise analysis that could benefit from having an alternative model to compare against. The larger argument hinges on the fact that the same frequencies are used to reconstructively model both the QBO and ENSO data sets. I don't know exactly how to quantify the likelihood of that being due to chance. BTW, I am getting criticisms from weird corners of the science blogosphere that assert that I am stealing these QBO and ENSO ideas from them without giving proper attribution.`

This is not exactly correct, but the odds of having the 2 fundamental frequencies of the QBO landing so close to the 2 main aliased lunar tidal frequencies just by chance is about 1 in 40,000. When the rather obvious plausibility of the physics is considered, this model is a winner. If I knew how to calculate it, both the BIC and AIC metric would probably be through the roof.

Then you take those same two fundamental frequencies and notice how close they map to core ENSO model forcing frequencies, it adds to the likelihood that this model is on the right track.

The statistical metrics are likely so good that (1) it becomes meaningless or (2) that people would suspect it as being rigged and so would dismiss it :) That's why I am waiting for Jan to add his input.

`This is not exactly correct, but the odds of having the 2 fundamental frequencies of the QBO landing so close to the 2 main aliased lunar tidal frequencies just by chance is about 1 in 40,000. When the rather obvious plausibility of the physics is considered, this model is a winner. If I knew how to calculate it, both the BIC and AIC metric would probably be through the roof. Then you take those same two fundamental frequencies and notice how close they map to core ENSO model forcing frequencies, it adds to the likelihood that this model is on the right track. The statistical metrics are likely so good that (1) it becomes meaningless or (2) that people would suspect it as being rigged and so would dismiss it :) That's why I am waiting for Jan to add his input.`

I still don't have the spare cycles to address this fully, but given that one of the two terms of an AIC or BIC is the log likelihood and there is not a closed form representation of the likelihood in this case, I'd probably explore either the empirical likelihood work of Art Owen and his students, for one thing as packaged in the

emplikR package, or possibly Approximate Bayesian Computation (ABC; see also and here).`I still don't have the spare cycles to address this fully, but given that one of the two terms of an AIC or BIC is the log likelihood and there is not a closed form representation of the likelihood in this case, I'd probably explore either the empirical likelihood work of [Art Owen](http://statweb.stanford.edu/~owen/empirical/) and his students, for one thing as [packaged in the _emplik_ R package](https://cran.r-project.org/web/packages/emplik/emplik.pdf), or possibly Approximate Bayesian Computation ([ABC](http://membres-timc.imag.fr/Olivier.Francois/CsilleryTREE10.pdf); [see also](https://cran.r-project.org/web/packages/abc/vignettes/abcvignette.pdf) and [here](http://med.stanford.edu/biostatistics/abstract/EBuzbas-reading01.pdf)).`

We have to be careful on how we define the probabilities. For one, it's not like we are dealing with a set of coins and randomly flipping to see if it comes up heads or tails. There is only one QBO and that is all we have in the statistical ensemble.

I can hazard a guess your idea would be to break this single time-series up into several intervals and then compare whether the QBO is positive or negative (or concave up/concave down) with respect to some null model such as a random walk over each interval.

Setting up a valid statistical and probabilistic premise for QBO has common origins with the historical story of Laplace and the Sunrise Problem.

And this gets weirder when the suggestion is also that the sun rising or not rising has more to do with it becoming a supernova on any given day (like stars are known to on occasion)

Yet since there is such a strong sense that the Draconic + Synodic/Sidereal aliased tide periods nail the underlying period of QBO so precisely, I can bet it will be no contest in this result. Its not much different than looking at tidal tables and learning over time that the numbers in those tables match the phases of the moon. In other words, much like in Laplace's sunrise problem, the totality of the evidence overwhelms any probability consideration. If we really need a model than we should look into how AIC or BIC confirms the connection between tides and the moon. I think it is essentially the same problem. Take a look at the recent model for the tidal equations. There are so many parameters in this set of equations that it would probably send any AIC or BIC metric into a tizzy.

If that is overkill, it all comes down to whether one believes a highly plausible physical model. The fact is that that Lindzen started working with his theory for QBO over 50 years ago and nowhere does his theory mention that the lunar forcing plays into it -- as in you can guess the underlying period from an aliased tidal period. Like I have said, it would be nutty if this concordance has been hiding in plain sight all these years.

`We have to be careful on how we define the probabilities. For one, it's not like we are dealing with a set of coins and randomly flipping to see if it comes up heads or tails. There is only one QBO and that is all we have in the statistical ensemble. I can hazard a guess your idea would be to break this single time-series up into several intervals and then compare whether the QBO is positive or negative (or concave up/concave down) with respect to some null model such as a random walk over each interval. Setting up a valid statistical and probabilistic premise for QBO has common origins with the historical story of Laplace and the [Sunrise Problem](https://en.wikipedia.org/wiki/Sunrise_problem). ![sunrise](https://i.ytimg.com/vi/G12jhgzCyJE/sddefault.jpg) > Laplace: "But this number [the probability of the sun coming up tomorrow] is far greater for him who, seeing in the totality of phenomena the principle regulating the days and seasons, realizes that nothing at present moment can arrest the course of it." And this gets weirder when the suggestion is also that the sun rising or not rising has more to do with it becoming a supernova on any given day (like stars are known to on occasion) Yet since there is such a strong sense that the Draconic + Synodic/Sidereal aliased tide periods nail the underlying period of QBO so precisely, I can bet it will be no contest in this result. Its not much different than looking at tidal tables and learning over time that the numbers in those tables match the phases of the moon. In other words, much like in Laplace's sunrise problem, the totality of the evidence overwhelms any probability consideration. If we really need a model than we should look into how AIC or BIC confirms the connection between tides and the moon. I think it is essentially the same problem. Take a look at the recent model for the [tidal equations](https://en.wikipedia.org/wiki/Theory_of_tides). There are so many parameters in this set of equations that it would probably send any AIC or BIC metric into a tizzy. If that is overkill, it all comes down to whether one believes a highly plausible physical model. The fact is that that Lindzen started working with his theory for QBO over 50 years ago and nowhere does his theory mention that the lunar forcing plays into it -- as in you can guess the underlying period from an aliased tidal period. Like I have said, it would be nutty if this concordance has been hiding in plain sight all these years.`

If the concern is that a particular series will come up with a specific AIC or BIC value, and there's no way to cross-validate, then the thing to do is use a bootstrap, in this case, a

stationarybootstrap(Politis and Romano) as is available in thetsbootstrapfunction of thetseriespackage of R. Each boot will return AIC and BIC values. As far as numbers of parameters go, if the equations have the same number of parameters, then models can be compared with one another by comparing the values of the information criteria, since the dependence-upon-number-of-parameters terms will cancel out. (If they don't, well, that's another thing, but don't need to indicate the number of parameters, just how they differ.) Also, when "so many parameters" is stated, are thesefreeparametersthat are adjusted for a fit?Also as far as Laplace and sunrise goes, AFAIC, if a philosophical or other argument brings someone to a place where they don't know what to do or cannot do anything, that counts as a failure of the argument in my book. Similarly, if there's a contradiction between "totality of evidence" and "probability consideration", IMO all that means is a malformed question or problem.

But anyway, it was asked, per #59 above.

`If the concern is that a particular series will come up with a specific AIC or BIC value, and there's no way to cross-validate, then the thing to do is use a bootstrap, in this case, a _stationary_ _bootstrap_ (Politis and Romano) as is available in the _tsbootstrap_ function of the _tseries_ package of R. Each boot will return AIC and BIC values. As far as numbers of parameters go, if the equations have the same number of parameters, then models can be compared with one another by comparing the values of the information criteria, since the dependence-upon-number-of-parameters terms will cancel out. (If they don't, well, that's another thing, but don't need to indicate the number of parameters, just how they differ.) Also, when "so many parameters" is stated, are these _free_ _parameters_ that are adjusted for a fit? Also as far as Laplace and sunrise goes, AFAIC, if a philosophical or other argument brings someone to a place where they don't know what to do or cannot do anything, that counts as a failure of the argument in my book. Similarly, if there's a contradiction between "totality of evidence" and "probability consideration", IMO all that means is a malformed question or problem. But anyway, it was asked, per #59 above.`

Jan, I don't have any free parameters in the model. These parameters are set by the lunar months and the agreement to these two values to within 0.015% was discovered by machine learning (see comments 323 and #58 which described independent ML experiments) . In other words, it was free from the influence of human bias, at least IMO.

So what I am saying is that the process is kind of like measuring that the rotation of the earth is 24 hours and then trying to use statistical metrics to correlate that timing to the sun rising in the morning. If that is just a coincidence ... I don't know what to say.

Keeping with the sunrise analogy, if we scale the discrepancy in terms of a 24-hour cycle, 0.015% is an error of only 13 seconds, and the fact that we have two of the primary lunar cycles both provide a match to within this margin is the model that has to be invalidated. And they both have to be there because the interference between the Draconic and Sidereal cycles leading to the well known tidal beat period of 18.6 years also has to be there as well -- which it certainly is, as the error in both cases is the same.

I hope that I don't come across as using an Argument From Personal Astonishment, but I honestly don't know how to proceed in quantifying the result as being there just by chance.

This is pretty useful to consider

As in the theory of tides, the forcing parameters are locked in place and there is little wiggle room.

If I made a mistake in how I set up the machine learning experiment or how to unalias the periods, perhaps it is more important that someone else try to reproduce the results, instead of diving deep into a statistical analysis. Yet, even then, if I did something wrong, it would be quite a coincidence to arrive at this result. This would be the biggest rabbit hole that I have ever descended into in my career. :)

`Jan, I don't have any free parameters in the model. These parameters are set by the lunar months and the agreement to these two values to within 0.015% was discovered by machine learning (see comments 323 and #58 which described independent ML experiments) . In other words, it was free from the influence of human bias, at least IMO. So what I am saying is that the process is kind of like measuring that the rotation of the earth is 24 hours and then trying to use statistical metrics to correlate that timing to the sun rising in the morning. If that is just a coincidence ... I don't know what to say. Keeping with the sunrise analogy, if we scale the discrepancy in terms of a 24-hour cycle, 0.015% is an error of only 13 seconds, and the fact that we have two of the primary lunar cycles both provide a match to within this margin is the model that has to be invalidated. And they both have to be there because the interference between the Draconic and Sidereal cycles leading to the well known tidal beat period of 18.6 years also has to be there as well -- which it certainly is, as the error in both cases is the same. I hope that I don't come across as using an Argument From Personal Astonishment, but I honestly don't know how to proceed in quantifying the result as being there just by chance. This is pretty useful to consider > "The solar tides are caused by thermal excitation primarily due to absorption of solar radiation by ozone in the stratosphere and water vapor in the troposphere. This heating is *almost independent of the motion of the atmosphere*, and it is periodic. The sun also imposes a periodic gravitational forcing that is completely independent of the motion of the atmosphere. Similarly, gravitational forcing due to the moon is periodic and completely independent of the motion. The theory of the tides seeks to explain how the observed tidal motions are driven by these externally imposed periodic forcings. The fact that the forcing functions for the tides are motion-independent and periodic explains why the theory of the tides is so well developed. It is much more difficult to solve the equations when the forcing (e.g. latent heat release) is dependent on the motion. .... Earlier we studied free oscillations. The tides are forced oscillations. For forced oscillations the period (or frequency) and zonal wave number of the forcing are *given*. In contrast, with free oscillations the frequency and zonal wave number of the solution are determined by solving an eigenvalue problem. " As in the theory of tides, the forcing parameters are locked in place and there is little wiggle room. If I made a mistake in how I set up the machine learning experiment or how to unalias the periods, perhaps it is more important that someone else try to reproduce the results, instead of diving deep into a statistical analysis. Yet, even then, if I did something wrong, it would be quite a coincidence to arrive at this result. This would be the biggest rabbit hole that I have ever descended into in my career. :)`

All I meant was that, if there aren't any free parameters to be fit, there's no correction to the negative log likelihood term in the AIC or BIC. The entire information criterion is then given by twice the negative log likelihood and, so, "There are so many parameters in this set of equations that it would probably send any AIC or BIC metric into a tizzy" is not consistent with what they are. So, accordingly, the likelihood function of the model tells all. If the model is too complicated so obtaining the likelihood function is not tractable, then it can (a) either be bounded, by describing simpler models with a likelihood strictly greater than and strictly less than the complicated one, or, (b), per Owen and colleagues, an estimate of the likelihood can be obtained empirically.

This is quite independent of the "machine learning experiment". Surely the ML will give a description of the empirical model it found, one that can be independently checked? Does the ML give an expression of how well it did? I'd be surprised it did not. It must have considered a number of alternatives when the model was built, and, so, have

somecriterion for preferring one to another. And the model, however good it is, cannot be perfect.Do you have numbers on the uncertainty in the measurements being used to assess the ML-found model?

`All I meant was that, if there aren't any free parameters to be fit, there's no correction to the negative log likelihood term in the AIC or BIC. The entire information criterion is then given by twice the negative log likelihood and, so, "There are so many parameters in this set of equations that it would probably send any AIC or BIC metric into a tizzy" is not consistent with what they are. So, accordingly, the likelihood function of the model tells all. If the model is too complicated so obtaining the likelihood function is not tractable, then it can (a) either be bounded, by describing simpler models with a likelihood strictly greater than and strictly less than the complicated one, or, (b), per Owen and colleagues, an estimate of the likelihood can be obtained empirically. This is quite independent of the "machine learning experiment". Surely the ML will give a description of the empirical model it found, one that can be independently checked? Does the ML give an expression of how well it did? I'd be surprised it did not. It must have considered a number of alternatives when the model was built, and, so, have _some_ criterion for preferring one to another. And the model, however good it is, cannot be perfect. Do you have numbers on the uncertainty in the measurements being used to assess the ML-found model?`

The machine learning via Eureqa uses an algorithm they call Symbolic Regression. For every analytic solution to the data, it gives an error metric and a complexity factor.

There is no question that Eureqa is finding the QBO

equivalentto the set of tidal equations that are used to model the periodicity of tides. But the issue is that as it finds the top two factors, that of the Draconic and Sidereal periods, with little complexity added, it then also tries to find the second-order terms (which are tabulated as tidal constituents) , and you can see the complexity start to sky-rocket. I do not have a good feel for how far to run with added complexity. We know that tidal equations are complex -- yet that complexity has been well validated by observations that match the theory. In other words, it is almost completely deterministic and non-chaotic, with the error only limited by the resolution in the data. No one seems to bat an eye over the fact that there are multiple sinusoidal terms in the tidal equations, yet with a new candidate model we have to tread carefully.I don't know when we will start to see diminishing returns as the noise in the QBO measurements start to creep in. In contrast, I have no doubt that Eureqa would be able to reconstruct the tidal equations from tidal data with very little effort, since that data is so clean and vast, but QBO is noisier and limited in scope.

Euereqa has an error metric they call logarithm error, which they say minimizes the squashed error log(1 + |error|). As I have been playing around with Eureqa for two-years now on this problem, I don't see much difference between using this and a non-log metric such as absolute error or correlation coefficient.

This back-and-forth is useful because we are obviously not in the same boat on what to do with the model or what it signifies. I look at the results of Eureqa and can immediately see the tidal connection as it zeros in on the tidal periods with startling precision. I think you are asking me to cast the error as a log-likelihood function.

$ \Sigma log(1 + |error|) $

I can certainly do that, but absent an alternative model to compare against, I don't know what it will tell us.

`The machine learning via Eureqa uses an algorithm they call Symbolic Regression. For every analytic solution to the data, it gives an error metric and a complexity factor. There is no question that Eureqa is finding the QBO *equivalent* to the set of tidal equations that are used to model the periodicity of tides. But the issue is that as it finds the top two factors, that of the Draconic and Sidereal periods, with little complexity added, it then also tries to find the second-order terms (which are tabulated as [tidal constituents](https://en.wikipedia.org/wiki/Theory_of_tides#Tidal_constituents)) , and you can see the complexity start to sky-rocket. I do not have a good feel for how far to run with added complexity. We know that tidal equations are complex -- yet that complexity has been well validated by observations that match the theory. In other words, it is almost completely deterministic and non-chaotic, with the error only limited by the resolution in the data. No one seems to bat an eye over the fact that there are multiple sinusoidal terms in the tidal equations, yet with a new candidate model we have to tread carefully. ![soln](http://imagizer.imageshack.us/a/img540/9429/S7WLD8.gif) I don't know when we will start to see diminishing returns as the noise in the QBO measurements start to creep in. In contrast, I have no doubt that Eureqa would be able to reconstruct the tidal equations from tidal data with very little effort, since that data is so clean and vast, but QBO is noisier and limited in scope. Euereqa has an error metric they call logarithm error, which they say minimizes the squashed error log(1 + |error|). As I have been playing around with Eureqa for two-years now on this problem, I don't see much difference between using this and a non-log metric such as absolute error or correlation coefficient. This back-and-forth is useful because we are obviously not in the same boat on what to do with the model or what it signifies. I look at the results of Eureqa and can immediately see the tidal connection as it zeros in on the tidal periods with startling precision. I think you are asking me to cast the error as a log-likelihood function. $ \Sigma log(1 + |error|) $ I can certainly do that, but absent an alternative model to compare against, I don't know what it will tell us.`

So each of those terms in the harmonic fit to the qbo is a "parameter" for the purposes of AIC and BIC, however Eureqa finds them. I count 17 parameters. I was wondering if there were any overfitting diagnostics? Can the facility do holdout cross validation on the data? Can it do a bootstrap? If not, one specific thing I could and will do, if you like, is that that harmonics model and the data, and do a proper cross-validation and see where it ends up.

To review, the purpose of cross-validation is to estimate the performance of the model on data not yet in hand or, if the series is non-stationary, if the world traced a different state trajectory for the same time.

I understand this can be complicated if "the data" actually consists of ensembles like HadCRUT4 is, which is the set I'm most familiar with. Indeed, using a set of ensembles has some subtle issues itself, which I've mentioned elsewhere, concerns that Daniel Wilks has expressed about such usage in his book,

StatisticalMethodsintheAtmosphericSciences(3rd edition), Section 7.7.1.`So each of those terms in the harmonic fit to the qbo is a "parameter" for the purposes of AIC and BIC, however Eureqa finds them. I count 17 parameters. I was wondering if there were any overfitting diagnostics? Can the facility do holdout cross validation on the data? Can it do a bootstrap? If not, one specific thing I could and will do, if you like, is that that harmonics model and the data, and do a proper cross-validation and see where it ends up. To review, the purpose of cross-validation is to estimate the performance of the model on data not yet in hand or, if the series is non-stationary, if the world traced a different state trajectory for the same time. I understand this can be complicated if "the data" actually consists of ensembles like HadCRUT4 is, which is the set I'm most familiar with. Indeed, using a set of ensembles has some subtle issues itself, which I've [mentioned elsewhere](https://johncarlosbaez.wordpress.com/2014/06/05/warming-slowdown-part-2/), concerns that Daniel Wilks has expressed about such usage in his book, _Statistical_ _Methods_ _in_ _the_ _Atmospheric_ _Sciences_ (3rd edition), Section 7.7.1.`

Here is an example of what detail that the machine learning is exposing with respect to second-order lunar terms in the QBO data.

In the previous comment, you can see a detected sinusoid of frequency 72.19 rads/year. The magnitude of this factor is less than the Draconic and Synodic/Sidereal factors but appears significant.

The connection is that 72.19 rads/year corresponds to a period of 31.79 days, which is awfully close to the evection period of the moon, 31.81 days. The evection period is a critical ingredient to formulating the Doodson variables in the tidal equation

The point is that each of the periods that the machine learning finds is related to lunar periods within a factor of 0.1%.

So when we count 17 parameters, are those parameters that match known physical quantities discounted in a complexity argument? Furthermore, the scaling parameters are really not that vital to a complexity argument because we know that the effects have to scale depending on the relative order of the forcing factors.

I want to argue with what Eureqa is finding, but when it offers it up on a silver platter like this, I will gladly run with it. That's why they called the tool Eureqa, i.e. if you apply it appropriately it exposes a gold mine of information :)

`Here is an example of what detail that the machine learning is exposing with respect to second-order lunar terms in the QBO data. In the previous comment, you can see a detected sinusoid of frequency 72.19 rads/year. The magnitude of this factor is less than the Draconic and Synodic/Sidereal factors but appears significant. The connection is that 72.19 rads/year corresponds to a period of 31.79 days, which is awfully close to the evection period of the moon, 31.81 days. The evection period is a critical ingredient to formulating the Doodson variables in the tidal equation ![evection](http://imageshack.com/a/img540/7194/3GCbpX.png) > from Marchuk, Guri I., and Boris Abramovich Kagan. Dynamics of ocean tides. Vol. 3. Springer Science & Business Media, 1989. ![evection2](http://imageshack.com/a/img911/629/LhwSnD.png) > from Bertotti, Bruno, Paolo Farinella, and David Vokrouhlický. Physics of the solar system: dynamics and evolution, space physics, and spacetime structure. Vol. 293. Springer Science & Business Media, 2003. The point is that each of the periods that the machine learning finds is related to lunar periods within a factor of 0.1%. So when we count 17 parameters, are those parameters that match known physical quantities discounted in a complexity argument? Furthermore, the scaling parameters are really not that vital to a complexity argument because we know that the effects have to scale depending on the relative order of the forcing factors. I want to argue with what Eureqa is finding, but when it offers it up on a silver platter like this, I will gladly run with it. That's why they called the tool Eureqa, i.e. if you apply it appropriately it exposes a gold mine of information :)`

Yes, but predictive accuracy does not in itself confirm correctness.

`Yes, but predictive accuracy does not in itself confirm correctness.`

I have a feeling the model is more likely correct than predictively accurate at this stage. I say that because correctness has to do with physical plausibility, whereas accurate predictions can occur if you just happen to discover a working set of heuristics.

How about turning the tables and suggest that someone has to successfully argue why lunar forcing

can notcause the QBO to oscillate as observed. If you think about it, that is a challenge that is as difficult to argue as defending why lunar forcingcan notcause ocean tides. The typical way of doing this is to show how the numbers don't match up. But since they do match in both cases it will take a different argument and one based on different physics. And that version of physics is likely overly complex in comparison to what I have, so it will lose when it comes to a complexity criteria showdown.Let me put that another way. Can you imagine a theory of tides that

didn'tinvolve the moon? Someone would likely have to concoct a system based on a modeled natural resonance that existed somewhere in the ocean. This resonance would have to be stationary over time, with the addition of intricate beat patterns that would need to be modeled as secondary resonances. Yet, that appears to be the kind of model that is currently in vogue for the QBO, building on the original QBO theory set forth by Richard Lindzen.All I can guess as to why an intuitive lunar forcing explanation has never been accepted is because a scientist with as large a contrarian streak as Lindzen (and as forceful a personality) has lorded over his own pet QBO theory the last 50 years. He is well known to concoct half-baked theories on climate science and perhaps people will start catching on to what he is all about. Thinking about the astronomer Thomas Gold as a comparison here. Otherwise the state of the science just doesn't make sense to me.

I am going out on a limb rationalizing here because I know these questions will eventually come if this new understanding ever begins to see the light of day outside of this forum. I have no other answer to the question of why is this model so late to the party.

`I have a feeling the model is more likely correct than predictively accurate at this stage. I say that because correctness has to do with physical plausibility, whereas accurate predictions can occur if you just happen to discover a working set of heuristics. How about turning the tables and suggest that someone has to successfully argue why lunar forcing *can not* cause the QBO to oscillate as observed. If you think about it, that is a challenge that is as difficult to argue as defending why lunar forcing *can not* cause ocean tides. The typical way of doing this is to show how the numbers don't match up. But since they do match in both cases it will take a different argument and one based on different physics. And that version of physics is likely overly complex in comparison to what I have, so it will lose when it comes to a complexity criteria showdown. Let me put that another way. Can you imagine a theory of tides that *didn't* involve the moon? Someone would likely have to concoct a system based on a modeled natural resonance that existed somewhere in the ocean. This resonance would have to be stationary over time, with the addition of intricate beat patterns that would need to be modeled as secondary resonances. Yet, that appears to be the kind of model that is currently in vogue for the QBO, building on the original QBO theory set forth by Richard Lindzen. All I can guess as to why an intuitive lunar forcing explanation has never been accepted is because a scientist with as large a contrarian streak as Lindzen (and as forceful a personality) has lorded over his own pet QBO theory the last 50 years. He is well known to concoct half-baked theories on climate science and perhaps people will start catching on to what he is all about. Thinking about the astronomer Thomas Gold as a comparison here. Otherwise the state of the science just doesn't make sense to me. I am going out on a limb rationalizing here because I know these questions will eventually come if this new understanding ever begins to see the light of day outside of this forum. I have no other answer to the question of why is this model so late to the party.`

Okay, then, so we're done. Good. Off to other things.

`Okay, then, so we're done. Good. Off to other things.`

Climate science is a fascinating field, filled with scientists that have forceful personalities.

This is what they say about Richard Lindzen in his Wikipedia entry

And then there is this interview with Lindzen called "Climate science is for second-raters says world's greatest atmospheric physicist"

You really have to wonder what is in Lindzen's head. Science is always about building on the work of others, so one should not denigrate those that came before you.

That is what I am struggling with right now. I am always looking for the simplest 1st-order physics I can apply to modeling a phenomenon. And this lunar tidal forcing model for QBO is as simple as it can get for modeling the basic periodicity of the historical time-series data. But will this not debunk or at least upend much of Lindzen's prior work? When Lindzen first started studying QBO in 1961, there were only 4 clear periods that had been measured since QBO monitoring began in 1953. Because the apparent value was thought to be 26 months instead of the 28 months as it is characterized now, Lindzen couldn't have associated the period with tidal values, as the numbers wouldn't have matched very well.

But with the benefit of hindsight and a long record of QBO data that has improved in quality over time, perhaps Lindzen's theory was premature and an alternate model is more viable. So when I say his models may be "half-baked", I mean that literally, in that they never had a time to gestate with greater amounts of data.

If I Google Scholar "QBO and Draconic". "QBO and Synodic" or any lunar-related topics, the only hits I get are to N. Sidorenkov, who is a Russian scientist that studies geophysical phenomena related to the Earth's rotation rate. I have been aware of his work for a couple of years now and I provided my cites to Ian Wilson who runs the AstroClimateConnection blog when he asked me last week if I was aware of that work. I had indeed cited Sidorenkov last year: http://contextearth.com/2015/09/04/the-qbom-part-2/#comments

No one in the scientific establishment is citing Sidorenkov. I have to say that I am likely building more off the work of Sidorenkov than someone like Lindzen. You go in the direction that the characterization takes you. So if Lindzen is a dead-end for further insight, others such as Sidorenkov can provide a fresher perspective.

`Climate science is a fascinating field, filled with scientists that have forceful personalities. This is what they say about Richard Lindzen in his Wikipedia entry > "Lindzen has been called a contrarian, in relation to climate change and other issues." And then there is this interview with Lindzen called ["Climate science is for second-raters says world's greatest atmospheric physicist"](http://blogs.telegraph.co.uk/news/jamesdelingpole/100257206/climate-science-is-for-second-raters-says-worlds-greatest-atmospheric-physicist/) > "I've asked very frequently at universities: 'Of the brightest people you know, how many people were studying climate [...or meteorology or oceanography...]?' And the answer is usually 'No one.'" > "You look at the credentials of some of these people [on the IPCC] and you realise that the world doesn't have that many experts, that many 'leading climate scientists'". > Was Lindzen suggesting, asked Tim Yeo at this point, that scientists in the field of climate were academically inferior. > "Oh yeah," said Lindzen. "I don't think there's any question that the brightest minds went into physics, math, chemistry…" You really have to wonder what is in Lindzen's head. Science is always about building on the work of others, so one should not denigrate those that came before you. That is what I am struggling with right now. I am always looking for the simplest 1st-order physics I can apply to modeling a phenomenon. And this lunar tidal forcing model for QBO is as simple as it can get for modeling the basic periodicity of the historical time-series data. But will this not debunk or at least upend much of Lindzen's prior work? When Lindzen first started studying QBO in 1961, there were only 4 clear periods that had been measured since QBO monitoring began in 1953. Because the apparent value was thought to be 26 months instead of the 28 months as it is characterized now, Lindzen couldn't have associated the period with tidal values, as the numbers wouldn't have matched very well. But with the benefit of hindsight and a long record of QBO data that has improved in quality over time, perhaps Lindzen's theory was premature and an alternate model is more viable. So when I say his models may be "half-baked", I mean that literally, in that they never had a time to gestate with greater amounts of data. If I Google Scholar "QBO and Draconic". "QBO and Synodic" or any lunar-related topics, the only hits I get are to N. Sidorenkov, who is a Russian scientist that studies geophysical phenomena related to the Earth's rotation rate. I have been aware of his work for a couple of years now and I provided my cites to Ian Wilson who runs the AstroClimateConnection blog when he asked me last week if I was aware of that work. I had indeed cited Sidorenkov last year: http://contextearth.com/2015/09/04/the-qbom-part-2/#comments No one in the scientific establishment is citing Sidorenkov. I have to say that I am likely building more off the work of Sidorenkov than someone like Lindzen. You go in the direction that the characterization takes you. So if Lindzen is a dead-end for further insight, others such as Sidorenkov can provide a fresher perspective.`

Here is the troubling aspect of using a lunar forcing model to characterize QBO. The current research has gotten to the point that climate scientists use large-scale simulations to model QBO [1].

I took some time to download the Community Atmosphere Model from here. This was a tarred-gzipped archive consisting of primarily Fortran modules. I grepped through the files looking for instances of the terms "draconic", "sidereal", "synodic", "lunar", "tidal", "tide", "moon". The only hits I got were to the term "diurnal", which appears to be related strictly to the diurnal solar cycle, and not the tidal diurnal cycle. The text does talk about gravity waves but these seem to be parameterized values in the code and not related to cyclic tidal gravity waves.

The fit of these models to the QBO data appears impressive, yet I do not think the CAM has any tie-ins to lunar forcing, and to get the right QBO period they tune the models with various atmospheric property terms . That's why I think a lunar-based forcing model is in a bind -- unless the consensus considers this as a valid factor, it really won't matter since the community will continue to use models that they think are correct. And those models do not come close to being able to do what Jan is requesting that I do with a simple model with no tunable factors to speak of.

[1]J. H. Richter, A. Solomon, and J. T. Bacmeister, “On the simulation of the quasi‐biennial oscillation in the Community Atmosphere Model, version 5,” Journal of Geophysical Research: Atmospheres, vol. 119, no. 6, pp. 3045–3062, 2014.

`Here is the troubling aspect of using a lunar forcing model to characterize QBO. The current research has gotten to the point that climate scientists use large-scale simulations to model QBO [1]. I took some time to download the Community Atmosphere Model from [here](http://www.cesm.ucar.edu/models/atm-cam/download/). This was a tarred-gzipped archive consisting of primarily Fortran modules. I grepped through the files looking for instances of the terms "draconic", "sidereal", "synodic", "lunar", "tidal", "tide", "moon". The only hits I got were to the term "diurnal", which appears to be related strictly to the diurnal solar cycle, and not the tidal diurnal cycle. The text does talk about gravity waves but these seem to be parameterized values in the code and not related to cyclic tidal gravity waves. The fit of these models to the QBO data appears impressive, yet I do not think the CAM has any tie-ins to lunar forcing, and to get the right QBO period they tune the models with various atmospheric property terms . That's why I think a lunar-based forcing model is in a bind -- unless the consensus considers this as a valid factor, it really won't matter since the community will continue to use models that they think are correct. And those models do not come close to being able to do what Jan is requesting that I do with a simple model with no tunable factors to speak of. [1]J. H. Richter, A. Solomon, and J. T. Bacmeister, “On the simulation of the quasi‐biennial oscillation in the Community Atmosphere Model, version 5,” Journal of Geophysical Research: Atmospheres, vol. 119, no. 6, pp. 3045–3062, 2014.`

As I am doing more background research into the history of QBO and how the consensus theory has developed, I noticed an ironic twist. The irony is that QBO theorist and noted AGW sceptic Richard Lindzen has lamented the missing "gentleman scientist", recently illustrated in the opening paragraph of this opinion piece.

It is only fitting that this new completely unfunded outsider model of QBO should on all accounts replace Lindzen's antiquated and much too complicated theory. I wonder if Lindzen will be enough of a "gentleman" to allow it to be considered next to his model?

Sorry for perhaps being overly dramatic, but the potential irony in all this is much too obvious to pass up. After studying the prior QBO research and always finding Lindzen mentioned as the originating theorist, but also as someone who doesn't have a grasp of AGW, it seems as if he may have set himself up.

Watch out Richard Lindzen -- that for which you wish for may come true.

On top of that, many of the other scientists who have done lots of what is considered the pioneering work on QBO, including Murry Salby, William Gray, and Peter Webster also hold strong views against mainstream climate sciences. Webster recently was quoted as saying

"You have signed the death warrant for science"in response to some minor incident. Salby doesn't even believe that the excess atmospheric CO2 is man-made! And William Gray has described global warming as a “hoax,” and something that “they've been brainwashing us [about] for 20 years.” Look it up on Google Scholar -- these are the researchers that have been most heavily cited regarding QBO.In various papers analyzing QBO:

Salby cited 152 times

Lindzen cited 498 times

Webster cited 358 times

Gray cited 746 times

These specific articles are all from American Meteorological Society journals. On the other hand, the evidence points to that QBO is not really a meteorological phenomenon, but is really geophysical.

The bottom-line is that I feel pretty good that I am on the right track, considering these people are considered the QBO experts. When preparing to defend your work, you first have to look at the competition. Those guys are all on the wrong side of the fence AFAIAC.

`As I am doing more background research into the history of QBO and how the consensus theory has developed, I noticed an ironic twist. The irony is that QBO theorist and noted AGW sceptic Richard Lindzen has lamented the missing "gentleman scientist", recently illustrated in the opening paragraph of this opinion piece. > [Science in the Public Square: Global Climate Alarmism and Historical Precedents](http://www.jpands.org/vol18no3/lindzen.pdf) > Journal of American Physicians and Surgeons Volume 18 Number 3 Fall 2013 > Richard S. Lindzen, Ph.D. > "Though valuable as a process, science is always problematic as an institution. Charles Darwin often expressed gratitude for being able to be a gentleman scientist with no need for an institutional affiliation. Unfortunately, as a practical matter, the gentleman scientist no longer exists. Even in the 19th Century, most scientists needed institutional homes, and today science almost inevitably requires outside funding. In some fields, including climate, the government has essentially a monopoly on such funding. > Expanded funding is eagerly sought, but the expansion of funding inevitably invites rent-seeking by scientists, university administration, and government bureaucracies. The public square brings its own dynamic into the process of science: most notably, it involves the coupling of science to specific policy issues. This is a crucial element in the climate issue, but comparable examples have existed in other fields, including eugenics and immigration, and Lysenkoism and agronomy" It is only fitting that this new completely unfunded outsider model of QBO should on all accounts replace Lindzen's antiquated and much too complicated theory. I wonder if Lindzen will be enough of a "gentleman" to allow it to be considered next to his model? Sorry for perhaps being overly dramatic, but the potential irony in all this is much too obvious to pass up. After studying the prior QBO research and always finding Lindzen mentioned as the originating theorist, but also as someone who doesn't have a grasp of AGW, it seems as if he may have set himself up. Watch out Richard Lindzen -- that for which you wish for may come true. On top of that, many of the other scientists who have done lots of what is considered the pioneering work on QBO, including Murry Salby, William Gray, and Peter Webster also hold strong views against mainstream climate sciences. Webster recently was quoted as saying *"You have signed the death warrant for science"* in response to some minor incident. Salby doesn't even believe that the excess atmospheric CO2 is man-made! And William Gray has described global warming as a “hoax,” and something that “they've been brainwashing us [about] for 20 years.” Look it up on Google Scholar -- these are the researchers that have been most heavily cited regarding QBO. In various papers analyzing QBO: * Salby cited [152 times](http://journals.ametsoc.org/doi/abs/10.1175/1520-0442(1999)012%3C2652:CBTSCA%3E2.0.CO;2) * Lindzen cited [498 times](http://journals.ametsoc.org/doi/abs/10.1175/1520-0469(1972)029%3C1076:AUTFTQ%3E2.0.CO;2) * Webster cited [358 times](http://journals.ametsoc.org/doi/abs/10.1175/1520-0469(2000)057%3C0613:LSDFAW%3E2.0.CO;2) * Gray cited [746 times](http://journals.ametsoc.org/doi/abs/10.1175/1520-0493(1984)112%3C1649:ASHFPI%3E2.0.CO;2) These specific articles are all from American Meteorological Society journals. On the other hand, the evidence points to that QBO is not really a meteorological phenomenon, but is really geophysical. The bottom-line is that I feel pretty good that I am on the right track, considering these people are considered the QBO experts. When preparing to defend your work, you first have to look at the competition. Those guys are all on the wrong side of the fence AFAIAC.`

Compared to the collection of scientists that I mentioned above, Kevin Trenberth seems like a sane voice. In this paper [1], he is discussing the link of stratospheric QBO to tropospheric (next step is ENSO).

The paper discusses aliasing and periodic elelments. This was published in 1980 but Trenberth's ideas were not followed through by others, AFAICT.

[1]K. F. Trenberth, “Atmospheric quasi-biennial oscillations,” Monthly Weather Review, vol. 108, no. 9, pp. 1370–1377, 1980. http://www.gps.caltech.edu/~kl/research/reading/Trenberth_MWR_1980.pdf

`Compared to the collection of scientists that I mentioned above, Kevin Trenberth seems like a sane voice. In this paper [1], he is discussing the link of stratospheric QBO to tropospheric (next step is ENSO). ![trenberth](http://contextearth.com/wp-content/uploads/2015/09/trenberthquote.png) The paper discusses aliasing and periodic elelments. This was published in 1980 but Trenberth's ideas were not followed through by others, AFAICT. [1]K. F. Trenberth, “Atmospheric quasi-biennial oscillations,” Monthly Weather Review, vol. 108, no. 9, pp. 1370–1377, 1980. http://www.gps.caltech.edu/~kl/research/reading/Trenberth_MWR_1980.pdf`

Is it interesting or just a coincidence that the fundamental period behind QBO and ENSO (2.33 years) matches that of the mean flood return period of 2.33 years as set as a reference standard by the USGS?

http://contextearth.com/2015/10/07/mean-flood-return-period-and-qbo-and-enso/

`Is it interesting or just a coincidence that the fundamental period behind QBO and ENSO (2.33 years) matches that of the mean flood return period of 2.33 years as set as a reference standard by the USGS? http://contextearth.com/2015/10/07/mean-flood-return-period-and-qbo-and-enso/`

That's pretty funny that after all this time, the machine learning fit finds this in the raw QBO data: When the QBO fit is un-aliased and extrapolated to an out-of-band interval, it matches the mean-square tidal potential [1].

http://contextearth.com/2015/10/09/qbo-is-a-lunar-solar-forced-system/

The data and machine learning speaks for itself -- this is nearly a hands-off computation. All I added was the unaliasing correction, which adds multiples of 2π (13×2π to be exact) to the fitted frequency terms.

Coincident with this, I uncovered a NASA JPL proposal that requested funding a collaboration effort titled "Importance of the Earth-Moon system for reducing uncertainties in climate modelling and monitoring". It discusses among other topics, unaliasing climate measures to understand potential lunar impacts. Greater detail on my blog post.

[1] R. D. Ray, “Decadal climate variability: Is there a tidal connection?,” Journal of climate, vol. 20, no. 14, pp. 3542–3560, 2007. PDF

`That's pretty funny that after all this time, the machine learning fit finds this in the raw QBO data: When the QBO fit is un-aliased and extrapolated to an out-of-band interval, it matches the mean-square tidal potential [1]. http://contextearth.com/2015/10/09/qbo-is-a-lunar-solar-forced-system/ ![ray](http://contextearth.com/wp-content/uploads/2015/10/qhIVmb1.gif) ![me](http://contextearth.com/wp-content/uploads/2015/10/WgSXK81.gif) The data and machine learning speaks for itself -- this is nearly a hands-off computation. All I added was the unaliasing correction, which adds multiples of 2π (13×2π to be exact) to the fitted frequency terms. Coincident with this, I uncovered a NASA JPL proposal that requested funding a collaboration effort titled "Importance of the Earth-Moon system for reducing uncertainties in climate modelling and monitoring". It discusses among other topics, unaliasing climate measures to understand potential lunar impacts. Greater detail on my blog post. [1] R. D. Ray, “Decadal climate variability: Is there a tidal connection?,” Journal of climate, vol. 20, no. 14, pp. 3542–3560, 2007. [PDF](http://journals.ametsoc.org/doi/pdf/10.1175/JCLI4193.1)`

Emergent properties are discovered with this fit

A bi-annual period is observed. The background is from R.D.Ray and has ~79 peaks in 40 years, while the model fit has ~78 peaks. Every other peak is suppressed, likely because of the $1/\omega^2$ suppression.

`Emergent properties are discovered with this fit ![b](http://imageshack.com/a/img908/8725/HjD0DX.gif) A bi-annual period is observed. The background is from R.D.Ray and has ~79 peaks in 40 years, while the model fit has ~78 peaks. Every other peak is suppressed, likely because of the $1/\omega^2$ suppression.`

From comment #76, I used machine learning to deconstruct the waveform from 1900 to 1900.15

Even though this interval is very short, it picks up frequencies close to the Draconic and Tropical lunar periods -- 168.2/2 and 41.71*2. With aliasing, multiples of 2 are indistinguishable.

The amazing aspect is how the nearly bi-annual period is recovered from the QBO data in comment #77. The machine learning algorithm is so discriminate and sensitive to be able to pick that up.

Is this all still statistically coincidental ? I have been looking at the work relating to fitting tidal models to actual data and they never seem to blink an eye about adding more and more harmonic components to make the fit more exact. The model of QBO may turn into a purely deterministic outcome of the lunar forcing in that case, and governed by the same well-known orbital periods. That is the way it is looking to me. There is no counter evidence yet to suggest that the QBO is not following the same pattern. And all it will take is for one conflict to break the model.

As you can see, a machine learning tool like Eureqa can deconstruct a tidal time series (even though it is very short), and so no wonder it can do the same with QBO.

`From comment #76, I used machine learning to deconstruct the waveform from 1900 to 1900.15 ![eur](http://imageshack.com/a/img912/3694/7h2WUU.jpg) Even though this interval is very short, it picks up frequencies close to the Draconic and Tropical lunar periods -- 168.2/2 and 41.71*2. With aliasing, multiples of 2 are indistinguishable. The amazing aspect is how the nearly bi-annual period is recovered from the QBO data in comment #77. The machine learning algorithm is so discriminate and sensitive to be able to pick that up. Is this all still statistically coincidental ? I have been looking at the work relating to fitting tidal models to actual data and they never seem to blink an eye about adding more and more harmonic components to make the fit more exact. The model of QBO may turn into a purely deterministic outcome of the lunar forcing in that case, and governed by the same well-known orbital periods. That is the way it is looking to me. There is no counter evidence yet to suggest that the QBO is not following the same pattern. And all it will take is for one conflict to break the model. As you can see, a machine learning tool like Eureqa can deconstruct a tidal time series (even though it is very short), and so no wonder it can do the same with QBO.`

The lunar forcing operates on so many time scales

From the QBO fit, the aliased 2.33 year period

In #76 you can see the 14 day repeat period fitted in #78

In #77 the 18.6 beat period

Next is the bi-annual period that seems to emerge from a combination of the other periods

The pattern seems complicated yet the machine learning can uncover it.

`The lunar forcing operates on so many time scales From the QBO fit, the aliased 2.33 year period In #76 you can see the 14 day repeat period fitted in #78 In #77 the 18.6 beat period Next is the bi-annual period that seems to emerge from a combination of the other periods ![ray](http://imageshack.com/a/img908/5427/tfxya1.png) The pattern seems complicated yet the machine learning can uncover it.`

Ray has a harmonic beat frequency of 18.61 yrs which is near to 8 of your 2.33 yr cycles at 18.64 yrs but not quite. Would more decimal places help and if so for which period?

`Ray has a harmonic beat frequency of 18.61 yrs which is near to 8 of your 2.33 yr cycles at 18.64 yrs but not quite. Would more decimal places help and if so for which period?`

Notice that in comment #79, the bi-annual peak has an irregular emphasis on every other peak. What I did was add 2π to the machine fitted 77.88 rads/yr component. This is fair since differences of 2π are indistinguishable to the gross fit (to the machine learning, it has to pick something and clearly it is seeing something at sub-monthly resolution). So now the component matches precisely with the averaged Draconic and Tropical month.

Now it is exactly a bi-annual match to the Ray tidal generated potential in black.

The exploded region still fits as well

There is a region highlighted in yellow which appears to be a gap in the data and looks suspiciously like a data entry error on Ray's part.

The fit operates on so many scales that the characterization is similar to what a cardiologist has to do when analyzing a pulse -- with the diastolic, systolic, etc rhythms.

`Notice that in comment #79, the bi-annual peak has an irregular emphasis on every other peak. What I did was add 2π to the machine fitted 77.88 rads/yr component. This is fair since differences of 2π are indistinguishable to the gross fit (to the machine learning, it has to pick something and clearly it is seeing something at sub-monthly resolution). So now the component matches precisely with the averaged Draconic and Tropical month. ![rayfit](http://imageshack.com/a/img908/851/j4ohDp.png) Now it is exactly a bi-annual match to the Ray tidal generated potential in black. The exploded region still fits as well ![rayfitshort](http://imageshack.com/a/img907/8027/S9wwVb.png) There is a region highlighted in yellow which appears to be a gap in the data and looks suspiciously like a data entry error on Ray's part. The fit operates on so many scales that the characterization is similar to what a cardiologist has to do when analyzing a pulse -- with the diastolic, systolic, etc rhythms.`

This may be a useful composite chart that I will continue to refer to, as it represents so well the causal forcing I am advancing for the QBO behavior. Open it up in another window if you want to see the detail.

It really has to be the moon. Things just don't align this well by happenstance.

The only liberty I took was in expanding the tidal axis from 1940 (Ray's original) to a little under 1941, so you can see the biannual alignment. I am not trying to cheat here because I want to expose the likely matching periodic factors instead of obscuring them.

`This may be a useful composite chart that I will continue to refer to, as it represents so well the causal forcing I am advancing for the QBO behavior. Open it up in another window if you want to see the detail. It really has to be the moon. Things just don't align this well by happenstance. ![qbo-extrap](http://imageshack.com/a/img911/3653/1VeW6C.png) ---- The only liberty I took was in expanding the tidal axis from 1940 (Ray's original) to a little under 1941, so you can see the biannual alignment. I am not trying to cheat here because I want to expose the likely matching periodic factors instead of obscuring them. ----`

Jim said:

I am not sure how much more I can do to make the model any more precise. This is my latest attempt, where I apply a close approximation of the tidal gravitational potential as a forcing for QBO

http://contextearth.com/2015/10/22/pukites-model-of-the-quasi-biennial-oscillation/

This is the most elegant and symmetrically pleasing model one can imagine and the fact that it lays right on top of the QBO .... what more can one do?

`Jim said: > "Ray has a harmonic beat frequency of 18.61 yrs which is near to 8 of your 2.33 yr cycles at 18.64 yrs but not quite. Would more decimal places help and if so for which period?" I am not sure how much more I can do to make the model any more precise. This is my latest attempt, where I apply a close approximation of the tidal gravitational potential as a forcing for QBO http://contextearth.com/2015/10/22/pukites-model-of-the-quasi-biennial-oscillation/ This is the most elegant and symmetrically pleasing model one can imagine and the fact that it lays right on top of the QBO .... what more can one do?`

I think it's pretty hawkeyed of you to find Ray's data and spot a possible error :). Maybe it's time for the final arxiv preprint? I'm still trying to catch up with Jan Galkowski's excellent stats references on cross-mapping (CCM) and Bayesian computation (ABC) methods.

`I think it's pretty hawkeyed of you to find Ray's data and spot a possible error :). Maybe it's time for the final arxiv preprint? I'm still trying to catch up with Jan Galkowski's excellent stats references on cross-mapping (CCM) and Bayesian computation (ABC) methods.`

Thanks.

I wish I could find Ray's exact algorithm for the potential. I was able to get close to what Ray had in his chart as you can see below. Note that even some of the beat patterns are reproduced faintly in the background.

I think I may have scared Jan away as he parted ways only saying he was "done". Information criteria metrics such as AIC and BIC require inputs for degrees of freedom in the model. The problem is that this QBO model has only one degree of freedom, and that is perhaps the amplitude scaling -- and that is not really a parameter because scaling is ignored in a correlation coefficient calculation.

So there is essentially no complexity to this model -- no degrees of freedom at this stage ! The only model assumption is that the tidal gravitational potential is aliased to the season. Everything else falls out of that premise.

Like I said in the post, this is a very mechanical recipe.

`Thanks. I wish I could find Ray's exact algorithm for the potential. I was able to get close to what Ray had in his chart as you can see below. Note that even some of the beat patterns are reproduced faintly in the background. ![ray](http://imageshack.com/a/img911/1373/OQ7bJP.png) I think I may have scared Jan away as he parted ways only saying he was "done". Information criteria metrics such as AIC and BIC require inputs for degrees of freedom in the model. The problem is that this QBO model has only one degree of freedom, and that is perhaps the amplitude scaling -- and that is not really a parameter because scaling is ignored in a correlation coefficient calculation. So there is essentially no complexity to this model -- no degrees of freedom at this stage ! The only model assumption is that the tidal gravitational potential is aliased to the season. Everything else falls out of that premise. Like I said in the post, this is a very mechanical recipe.`

I too found Jan's statement to the effect that "good, we're done then" enigmatic. And as for AIC I assumed that whatever the current best of breed prediction for the SOI would be the comparator.

`I too found Jan's statement to the effect that "good, we're done then" enigmatic. And as for AIC I assumed that whatever the current best of breed prediction for the SOI would be the comparator.`

Jim, Yes, definitely the SOI model will need something to compare against, but this QBO model is looking good at the moment on its own terms. I do think the two behaviors are intimately tied, because SOI does appear to require the same lunar forcing as QBO, plus some lower frequency forcing.

Here is also a prescient and totally not enigmatic quote I found from Richard Lindzen concerning possible lunar forcing on QBO:

I can't believe he stated all that yet could not find the connection to the lunar tidal after all these years. It is entirely possible that the

"sufficiently long records"stopped him from observing the correlation. Or possibly the fact that aliasing wasn't understood.I hadn't read that passage when I wrote this in the blog post:

The gist in what Lindzen said and what I am saying is that this is such a simple system construct, that if it works then it is "unlikely that lunar periods could be produced by anything other than the lunar potential" . I really don't even need to formulate the physical argument, since Lindzen already stated it over 40 years ago. Perhaps Lindzen will need to take his own medicine on this one.

And consider the prescient line:

"if theory is incapable of explaining observations for such a simple system, we may plausibly be concerned with our ability to explain more complicated systems." That statement is jaw dropping if one looks at the complexity in the QBO models that has come since that was written in 1974. Everyone must have just assumed that the simplest model would not work. That is the only sense I can make of this.`Jim, Yes, definitely the SOI model will need something to compare against, but this QBO model is looking good at the moment on its own terms. I do think the two behaviors are intimately tied, because SOI does appear to require the same lunar forcing as QBO, plus some lower frequency forcing. Here is also a prescient and totally not enigmatic quote I found from Richard Lindzen concerning possible lunar forcing on QBO: >" *5. Lunar semidiurnal tide* : One rationale for studying tides is that they are motion systems for which we know the periods perfectly, and the forcing almost as well (this is certainly the case for gravitational tides). Thus, it is relatively easy to isolate tidal phenomena in the data, to calculate tidal responses in the atmosphere, and to compare the two. Briefly, conditions for comparing theory and observation are relatively ideal. Moreover, if theory is incapable of explaining observations for such a simple system, we may plausibly be concerned with our ability to explain more complicated systems. Lunar tides are especially well suited to such studies since it is unlikely that lunar periods could be produced by anything other than the lunar tidal potential. The only drawback in observing lunar tidal phenomena in the atmosphere is their weak amplitude, but with sufficiently long records this problem can be overcome [viz. discussion in Chapman and Lindaen (1970)] at least in analyses of the surface pressure oscillation. " -- from *Lindzen, Richard S., and Siu-Shung Hong. "Effects of mean winds and horizontal temperature gradients on solar and lunar semidiurnal tides in the atmosphere." Journal of the Atmospheric Sciences 31.5 (1974): 1421-1446*. I can't believe he stated all that yet could not find the connection to the lunar tidal after all these years. It is entirely possible that the *"sufficiently long records"* stopped him from observing the correlation. Or possibly the fact that aliasing wasn't understood. I hadn't read that passage when I wrote this in the blog post: >"If one then matches this plot against the QBO time-series, you will find a high correlation coefficient. If the lunar potential is tweaked away from its stationary set of parameters, the fit degrades rapidly. So it becomes essentially a binary match. If it didn't fit, then the lunar gravitational potential hypothesis would be invalidated. But since it does fit precisely, then it remains a highly plausible model." The gist in what Lindzen said and what I am saying is that this is such a simple system construct, that if it works then it is "unlikely that lunar periods could be produced by anything other than the lunar potential" . I really don't even need to formulate the physical argument, since Lindzen already stated it over 40 years ago. Perhaps Lindzen will need to take his own medicine on this one. And consider the prescient line: *"if theory is incapable of explaining observations for such a simple system, we may plausibly be concerned with our ability to explain more complicated systems*." That statement is jaw dropping if one looks at the complexity in the QBO models that has come since that was written in 1974. Everyone must have just assumed that the simplest model would not work. That is the only sense I can make of this.`

Getting back to what started this thread on the behavior of QBO, Graham Jones was puzzling over why climatologists seem to think that QBO is only predictable over a short time span, in contrast to what I had been asserting.

I had planted the seed by pointing out that anyone can look at the QBO time-series data and see that it is periodic, with some frequency modulation and some noise as well.

As I drilled down, what I tried to demonstrate is that QBO remarkably follows precisely the pattern of the periodic lunar tide potential with a similar frequency modulation at the same scale (~18 years).

Yet, no research, including that of Richard Lindzen -- who actually considered the lunar tide potential -- has shown the same correlation.

This is very concerning because any paper I write on these findings will directly contradict the current research results.

I say QBO is periodic ... but they (climate science literature) say it is unpredictable.

I say QBO follows the lunar forcing ... but they will say the forcing varies by another mechanism.

I get the feeling that for anything I write, the response will be -- Who are you going to believe, us climate scientists or your lying eyes?

If anyone remembers Dara O'Shayda commenting in this forum, recall his frustration with the entrenched viewpoints in earth sciences. I have not yet reached Dara's threshold but I don't know how best to challenge the current thinking without sounding shrill.

Bottomline, what I am advocating is really no different than the suggestion that the up-and-down motion of ocean tides follow the phases of the moon and the sun. It is really that straightforward and no one questions that the moon causes the tides. I am just applying the advice of what Richard Lindzen gave in 1974 --

"Lunar tides are especially well suited to such studies since it is unlikely that lunar periods could be produced by anything other than the lunar tidal potential."Remember, this is all data-driven, there are no free parameters, and so there is no human bias in manipulating this correlation. Machine learning verifies the correlation. Others can as well. That's all there is to it.

I hope not to alienate more people, like I apparently alienated Jan Galkowski, but its getting to the point that I have to resort to sociology arguments to figure out how to proceed.

`Getting back to what started this thread on the behavior of QBO, Graham Jones was puzzling over why climatologists seem to think that QBO is only predictable over a short time span, in contrast to what I had been asserting. I had planted the seed by pointing out that anyone can look at the QBO time-series data and see that it is periodic, with some frequency modulation and some noise as well. As I drilled down, what I tried to demonstrate is that QBO remarkably follows precisely the pattern of the periodic lunar tide potential with a similar frequency modulation at the same scale (~18 years). Yet, no research, including that of Richard Lindzen -- who actually considered the lunar tide potential -- has shown the same correlation. This is very concerning because any paper I write on these findings will directly contradict the current research results. 1. I say QBO is periodic ... but they (climate science literature) say it is unpredictable. 2. I say QBO follows the lunar forcing ... but they will say the forcing varies by another mechanism. I get the feeling that for anything I write, the response will be -- Who are you going to believe, us climate scientists or your lying eyes? If anyone remembers Dara O'Shayda commenting in this forum, recall his frustration with the entrenched viewpoints in earth sciences. I have not yet reached Dara's threshold but I don't know how best to challenge the current thinking without sounding shrill. Bottomline, what I am advocating is really no different than the suggestion that the up-and-down motion of ocean tides follow the phases of the moon and the sun. It is really that straightforward and no one questions that the moon causes the tides. I am just applying the advice of what Richard Lindzen gave in 1974 -- *"Lunar tides are especially well suited to such studies since it is unlikely that lunar periods could be produced by anything other than the lunar tidal potential."* Remember, this is all data-driven, there are no free parameters, and so there is no human bias in manipulating this correlation. Machine learning verifies the correlation. Others can as well. That's all there is to it. I hope not to alienate more people, like I apparently alienated Jan Galkowski, but its getting to the point that I have to resort to sociology arguments to figure out how to proceed.`

The first chart above is a model of the QBO that uses the same principal lunar potential frequencies applied in tidal harmonic analysis, but aliased to one year boundaries. Note the richness in the detail of the QBO that the model is matching. The key here is not to filter the QBO data at all -- much of the sub-year detail is captured by the higher harmonics of the one-year aliased fundamental.

The second is a back extrapolation, which shows the 18.6 year beat frequency in the lunar nodes.

Richard Ray of NASA Goddard published this table of rotation rate coefficients last year. What commodity tide matching algorithms do is pick the strongest gravitational coefficients from the table to get a good first-order match to measured tidal gauge data. They apparently don't blink an eye over what looks like complexity in a model fit. All they have to say is that this is the known physics of the orbit of the moon with the earth and the earth/moon with the sun and the rest is essentially fill-in-the-blanks on what the magnitude of the individual factors are. So I follow their basic recipe and apply a multiple linear regression of the strongest factors ( $V_0/g$ ) to get the fit above.

Again, I have no idea why this kind of analysis was not performed over the last few decades in which this data has been available. Remember that Richard Lindzen has been studying the QBO for over 50 years, and he even said this in 1974 --

"Lunar tides are especially well suited to such studies since it is unlikely that lunar periods could be produced by anything other than the lunar tidal potential."Nice that he wrote this, as any critical response to these new findings really have to be prefaced with Lindzen's words. Blame it all on him. :)

If that isn't enough, the model of ENSO that I have been working on uses these same factors, but weighted to longer periods as the natural response of the ocean damps out faster cycles.

`![QBO fit](http://imageshack.com/a/img907/2687/cQCRXc.png) ![QBO extend](http://imageshack.com/a/img910/3743/xVP4Of.png) The first chart above is a model of the QBO that uses the same principal lunar potential frequencies applied in tidal harmonic analysis, but aliased to one year boundaries. Note the richness in the detail of the QBO that the model is matching. The key here is not to filter the QBO data at all -- much of the sub-year detail is captured by the higher harmonics of the one-year aliased fundamental. The second is a back extrapolation, which shows the 18.6 year beat frequency in the lunar nodes. Richard Ray of NASA Goddard published this table of rotation rate coefficients last year. What commodity tide matching algorithms do is pick the strongest gravitational coefficients from the table to get a good first-order match to measured tidal gauge data. They apparently don't blink an eye over what looks like complexity in a model fit. All they have to say is that this is the known physics of the orbit of the moon with the earth and the earth/moon with the sun and the rest is essentially fill-in-the-blanks on what the magnitude of the individual factors are. So I follow their basic recipe and apply a multiple linear regression of the strongest factors ( $V_0/g$ ) to get the fit above. ![table](http://imageshack.com/a/img911/6434/l5UNiq.gif) Again, I have no idea why this kind of analysis was not performed over the last few decades in which this data has been available. Remember that Richard Lindzen has been studying the QBO for over 50 years, and he even said this in 1974 -- *"Lunar tides are especially well suited to such studies since it is unlikely that lunar periods could be produced by anything other than the lunar tidal potential."* Nice that he wrote this, as any critical response to these new findings really have to be prefaced with Lindzen's words. Blame it all on him. :) If that isn't enough, the model of ENSO that I have been working on uses these same factors, but weighted to longer periods as the natural response of the ocean damps out faster cycles.`

This is really cool, as I take the second derivative of the QBO data and then fit the set of aliased lunar tidal cycles via multiple linear regression. Taking a derivative adds lots of noise to the data, so taking another derivative beyond that just multiplies the noise. Yet the intricate detail that is retained in the model fit is striking.

Why take the second derivative? Because that enables a better match to the wave equation without having to do an integration. This model then becomes very close to the forced response solution to the wave equation of atmospheric winds.

`![2nd](http://imageshack.com/a/img910/2447/EhIOPp.png) This is really cool, as I take the second derivative of the QBO data and then fit the set of aliased lunar tidal cycles via multiple linear regression. Taking a derivative adds lots of noise to the data, so taking another derivative beyond that just multiplies the noise. Yet the intricate detail that is retained in the model fit is striking. Why take the second derivative? Because that enables a better match to the wave equation without having to do an integration. This model then becomes very close to the forced response solution to the wave equation of atmospheric winds.`

This is a representative table of the forcing factors that go into the prior three figures. One for

f(t)and one forf''(t). The latter is more sensitive to the higher forcing frequencies, as the fine sub-yearly structure is expose on taking the second derivative. The main factor again is the Draconic or Nodal aliased lunar month forcing. The only questionable factor is the 2.96 year period, which I haven't been able to identify from the known lunar tidal factors but is possibly related to a spin-orbit coupling perturbation in the moon's rotation, which I have seen cited as 2.9 years. At this stage 2.9 vs 2.96 is actually a significant difference. Yet, this unknown 2.96 factor is only the 10th strongest factor and doesn't actually make a big difference in the overall fit.`This is a representative table of the forcing factors that go into the prior three figures. One for *f(t)* and one for *f''(t)*. The latter is more sensitive to the higher forcing frequencies, as the fine sub-yearly structure is expose on taking the second derivative. The main factor again is the Draconic or Nodal aliased lunar month forcing. The only questionable factor is the 2.96 year period, which I haven't been able to identify from the known lunar tidal factors but is possibly related to a spin-orbit coupling perturbation in the moon's rotation, which I have seen cited as 2.9 years. At this stage 2.9 vs 2.96 is actually a significant difference. Yet, this unknown 2.96 factor is only the 10th strongest factor and doesn't actually make a big difference in the overall fit. ![tab](http://imageshack.com/a/img908/1160/CDhROP.gif)`

The current QBO model only uses the exact periods from the lunar tables, and it can generate very good out-of-band projections based on training intervals.

This is fitting to the noisy 2nd derivative of the 30 hPa QBO data:

The training is up to year 2000 using multiple linear regression. There is also much more structure to the 2nd derivative, yet you can see how the projection beyond 2000 finds that structure. The structure is actually composed of precise harmonics of the primary aliased lunar nodal cycle. There is a deterministic rhythm to the time-profile that works exactly like that for ocean tides, but impacted by the strong seasonal signal -- which doesn't happen with tides and therefore the overall cycle doesn't look like a tide chart, as it aliases to a completely different fundamental frequency.

Here is the backcast version which uses data from 1975 to 2015, and then works backward

Yesterday I find that Gavin Schmidt and company at NASA Goddard have written a lengthy paper on how to generate a model of the QBO [1]. I can't seem to find anything remotely trivial in what they are doing. That's 26 pages of stuff that I am not going to try to figure out. Thanks to Richard Lindzen for leading everyone astray on QBO for 40+ years :(

[1]D. Rind, J. Jonas, N. Balachandran, G. Schmidt, and J. Lean, “The QBO in two GISS global climate models: 1. Generation of the QBO,” Journal of Geophysical Research: Atmospheres, vol. 119, no. 14, pp. 8798–8824, 2014.

`The current QBO model only uses the exact periods from the lunar tables, and it can generate very good out-of-band projections based on training intervals. This is fitting to the noisy 2nd derivative of the 30 hPa QBO data: ![trainfore](http://imageshack.com/a/img903/9871/u5IiHE.png) The training is up to year 2000 using multiple linear regression. There is also much more structure to the 2nd derivative, yet you can see how the projection beyond 2000 finds that structure. The structure is actually composed of precise harmonics of the primary aliased lunar nodal cycle. There is a deterministic rhythm to the time-profile that works exactly like that for ocean tides, but impacted by the strong seasonal signal -- which doesn't happen with tides and therefore the overall cycle doesn't look like a tide chart, as it aliases to a completely different fundamental frequency. Here is the backcast version which uses data from 1975 to 2015, and then works backward ![back](http://imageshack.com/a/img903/6489/kSKbeF.png) Yesterday I find that Gavin Schmidt and company at NASA Goddard have written a lengthy paper on how to generate a model of the QBO [1]. I can't seem to find anything remotely trivial in what they are doing. That's 26 pages of stuff that I am not going to try to figure out. Thanks to Richard Lindzen for leading everyone astray on QBO for 40+ years :( [1]D. Rind, J. Jonas, N. Balachandran, G. Schmidt, and J. Lean, “The QBO in two GISS global climate models: 1. Generation of the QBO,” Journal of Geophysical Research: Atmospheres, vol. 119, no. 14, pp. 8798–8824, 2014.`

This is the 2nd-derivative model fit for QBO at 70 hPa which is the lowest atmospheric reading. Just about every peak and valley aligns, and the only question is the relative strength and whether it appears as a clear peak or as a shoulder.

The model may still have one missing factor that isn't showing up, or it may be a good idea to solve the full wave equation, which is what works well for the ENSO model.

`This is the 2nd-derivative model fit for QBO at 70 hPa which is the lowest atmospheric reading. Just about every peak and valley aligns, and the only question is the relative strength and whether it appears as a clear peak or as a shoulder. ![second](http://imageshack.com/a/img910/9383/OIoaLA.png) The model may still have one missing factor that isn't showing up, or it may be a good idea to solve the full wave equation, which is what works well for the ENSO model.`

We use seasonal aliasing of the lunar gravitational pull to generate the incredible fits to the QBO

How does one get the harmonics of the aliasing?

The starting premise is that a known lunar tidal forcing signal is periodic

$$L(t) = k \cdot sin(\omega_L t + \phi) $$ The seasonal signal is likely a strong periodic delta function, which peaks at a specific time of the year. This can be approximated as a Fourier series of period $2\pi$.

$$s(t) = \sum\limits_{i=1}^n a_i sin(2 \pi t i +\theta_i) $$ For now, the exact form of this doesn't matter, as what we are trying to show is how the aliasing comes about.

The forcing is then a combination of the lunar cycles $L(t)$ amplified in some way by the strongly cyclically peaked seasonal signal $s(t)$.

$$f(t) = s(t) L(t)$$ Multiplying this out, and pulling the lunar factor into the sum

$$f(t) = k \sum\limits_{i=1}^n a_i sin(\omega_L t + \phi) sin(2 \pi t i +\theta_i)$$ then with the trig identity

$$sin(x) sin(y) = \frac{1}{2} (cos(x-y)-cos(x+y))$$ Expanding the lower frequency difference terms and ignoring the higher frequency additive terms

$$f(t) = k/2 \sum\limits_{i=1}^n a_i sin((\omega_L - 2 \pi i)t +\psi_i) + ... $$ Now you can see how the high frequency $\omega_L$ term gets reduced in frequency by multiples of $2\pi$, until it nears the period of the seasonal cycle. And those are the factors that feature in the multiple regression fit. What the regression does is determine the weighting of the $a_i$ terms, across the set of lunar $\omega_L$ terms.

This is a link to the interactive QBO app on the dynamic context server: http://entroplet.com/context_qbo/navigate

`We use seasonal aliasing of the lunar gravitational pull to generate the incredible fits to the QBO How does one get the harmonics of the aliasing? The starting premise is that a known lunar tidal forcing signal is periodic $$L(t) = k \cdot sin(\omega_L t + \phi) $$ The seasonal signal is likely a strong periodic delta function, which peaks at a specific time of the year. This can be approximated as a Fourier series of period $2\pi$. $$s(t) = \sum\limits_{i=1}^n a_i sin(2 \pi t i +\theta_i) $$ For now, the exact form of this doesn't matter, as what we are trying to show is how the aliasing comes about. The forcing is then a combination of the lunar cycles $L(t)$ amplified in some way by the strongly cyclically peaked seasonal signal $s(t)$. $$f(t) = s(t) L(t)$$ Multiplying this out, and pulling the lunar factor into the sum $$f(t) = k \sum\limits_{i=1}^n a_i sin(\omega_L t + \phi) sin(2 \pi t i +\theta_i)$$ then with the trig identity $$sin(x) sin(y) = \frac{1}{2} (cos(x-y)-cos(x+y))$$ Expanding the lower frequency difference terms and ignoring the higher frequency additive terms $$f(t) = k/2 \sum\limits_{i=1}^n a_i sin((\omega_L - 2 \pi i)t +\psi_i) + ... $$ Now you can see how the high frequency $\omega_L$ term gets reduced in frequency by multiples of $2\pi$, until it nears the period of the seasonal cycle. And those are the factors that feature in the multiple regression fit. What the regression does is determine the weighting of the $a_i$ terms, across the set of lunar $\omega_L$ terms. This is a link to the interactive QBO app on the dynamic context server: http://entroplet.com/context_qbo/navigate`

Why the Quasi-Biennial Oscillation matters

`<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">Here is how a model of Quasi-Biennial Oscillations <a href="https://twitter.com/hashtag/QBO?src=hash">#QBO</a> figures into <a href="https://twitter.com/google">@Google</a>'s <a href="https://twitter.com/hashtag/ProjectLoon?src=hash">#ProjectLoon</a> <a href="https://t.co/FgYpR3ZM7I">https://t.co/FgYpR3ZM7I</a> <a href="https://t.co/OBPVlLirt4">pic.twitter.com/OBPVlLirt4</a></p>— Paul Pukite (@WHUT) <a href="https://twitter.com/WHUT/status/667755157069824001">November 20, 2015</a></blockquote> <script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script> ![pic](http://images.hngn.com/data/images/full/93252/project-loon.jpg) [Why the Quasi-Biennial Oscillation matters](http://www.ecmwf.int/en/about/media-centre/news/2015/why-quasi-biennial-oscillation-matters) >"The poor representation of the QBO in climate change models means that no-one knows what will happen to the QBO in the decades ahead – will it remain largely unchanged, will its period lengthen, or will it change more radically? There is also increasing interest in stratospheric wind forecasts in their own right: Google has a programme, dubbed Project Loon, to provide global internet coverage using actively controlled stratospheric balloons. This both requires and contributes to data on stratospheric winds, with forecasts being needed on various timescales to control the system."`

I might as well put this thing to bed with this next chart :) There are no adjustable parameters in my QBO model, as it relies completely on the known lunar tidal periods. So it's actually difficult to do a real sensitivity analysis. What I thought to do is consider how to move the periods in unison away from the nominal values.

One clever way of doing this is to change a common factor used in the aliasing calculation for each of these periods. Obviously the best candidate is the solar year. To get the correct aliasing with respect to each of the periods, the value of the year in days must be set (see comment #94 ). And because of the aliased signal's period sensitivity to the mixing frequency, any change in the year value will magnify the error in a fit. So if the theoretically predicted aliased lunar months form an optimal fit, then any change in the solar year away from the nominal value will degrade this fit. And it should degrade it rapidly, because of the sensitivity of the aliasing calculation.

In the following chart, the nominal value for the solar year is 365.242 days, and the x-axis shows what happens when the value is modified away from this value, based on the multiple linear regression fit. The upper-right inset is a magnified region about 365 days.

The best fit is 365.23, which is close enough to 365.242 in my book.

`I might as well put this thing to bed with this next chart :) There are no adjustable parameters in my QBO model, as it relies completely on the known lunar tidal periods. So it's actually difficult to do a real sensitivity analysis. What I thought to do is consider how to move the periods in unison away from the nominal values. One clever way of doing this is to change a common factor used in the aliasing calculation for each of these periods. Obviously the best candidate is the solar year. To get the correct aliasing with respect to each of the periods, the value of the year in days must be set (see [comment #94](https://forum.azimuthproject.org/discussion/comment/15004/#Comment_15004) ). And because of the aliased signal's period sensitivity to the mixing frequency, any change in the year value will magnify the error in a fit. So if the theoretically predicted aliased lunar months form an optimal fit, then any change in the solar year away from the nominal value will degrade this fit. And it should degrade it rapidly, because of the sensitivity of the aliasing calculation. In the following chart, the nominal value for the solar year is 365.242 days, and the x-axis shows what happens when the value is modified away from this value, based on the multiple linear regression fit. The upper-right inset is a magnified region about 365 days. ![chart](http://imageshack.com/a/img911/5355/poz1WD.png) The best fit is 365.23, which is close enough to 365.242 in my book.`

ML stuff

http://ContextEarth.com/2015/12/02/eureqa/

http://ContextEarth.com/2015/12/03/daily-double/

`ML stuff http://ContextEarth.com/2015/12/02/eureqa/ http://ContextEarth.com/2015/12/03/daily-double/`

Over at the Open Mind blog, Tamino has written a piece exposing Richard Lindzen's manipulations.

Other commenters have piled on and provided examples of the strange ideas that Lindzen has promoted over the years, such as the idea of Mars warming in conjunction with the Earth.

The more I read about Lindzen's scientific approach, the more convinced I am that his theory of QBO is not applicable to the actual behavior that is observed. Cyclic phenomena such as QBO do not spontaneously oscillate with such precision unless a driving force with the same period is causing the alignment. Somehow Lindzen convinced everyone 40 years ago that a complicated narrative of atmospheric flows would cause the 28-month period of QBO to appear.

I really don't understand how he snookered his peers into believing such a theory when a perfectly good explanation by way of lunisolar gravitational potential is available. The seasonally aliased nodal, anomalistic, and tropical lunar cycles, when combined according to their relative strengths, becomes a perfect candidate for QBO forcing.

When the correlation coefficient is greater for out-of-band testing than it is for a training interval, you discount the idea that it is just the result of overfitting. See this chart of the 2nd-derivative of QBO:

To invalidate the lunisolar gravitational forcing theory for QBO, all one has to show is that the numbers do not align with observation. Is anyone else willing to show that they don't align?

`Over at the Open Mind blog, Tamino has [written a piece exposing Richard Lindzen's manipulations](https://tamino.wordpress.com/2015/12/26/richard-lindzen-limited-understanding/#comment-92540). Other commenters have piled on and provided examples of the strange ideas that Lindzen has promoted over the years, such as the idea of Mars warming in conjunction with the Earth. The more I read about Lindzen's scientific approach, the more convinced I am that his theory of QBO is not applicable to the actual behavior that is observed. Cyclic phenomena such as QBO do not spontaneously oscillate with such precision unless a driving force with the same period is causing the alignment. Somehow Lindzen convinced everyone 40 years ago that a complicated narrative of atmospheric flows would cause the 28-month period of QBO to appear. I really don't understand how he snookered his peers into believing such a theory when a perfectly good explanation by way of lunisolar gravitational potential is available. The seasonally aliased nodal, anomalistic, and tropical lunar cycles, when combined according to their relative strengths, becomes a perfect candidate for QBO forcing. When the correlation coefficient is greater for out-of-band testing than it is for a training interval, you discount the idea that it is just the result of overfitting. See this chart of the 2nd-derivative of QBO: ![qbo](http://imageshack.com/a/img903/9871/u5IiHE.png) To invalidate the lunisolar gravitational forcing theory for QBO, all one has to show is that the numbers do not align with observation. Is anyone else willing to show that they don't align?`

This climate research group[1] claims that biennial oscillations are indistinguishable from white noise

http://www.researchgate.net/publication/281899198_Tropospheric_Biennial_Oscillation_TBO_indistinguishable_from_white_noise

[1]M. F. Stuecker, A. Timmermann, J. Yoon, and F. Jin, “Tropospheric Biennial Oscillation (TBO) indistinguishable from white noise,” Geophysical Research Letters, vol. 42, no. 18, pp. 7785–7791, 2015.

They see the probability of detecting two-year oscillation periods in the fluctuations of white noise at 2/3. For discrete coin flipping, the probability has to be locked at 1/2, but since climate measures are continuous, the arithmetic of PDFs will weigh the results toward a flip in direction every year.

I can see how this can occur : Say a value is at 0.8 in a range of 0 to 1 for one particular year. For white noise, the value can be anything so there is an 80% chance that it will be lower the next year. That change in direction from a high value to a low value (or vice versa) constitutes a biennial oscillation from their point-of-view,

What they are trying to do is marginalize the idea of Meehl[2] and others[3,4] who claim that Tropospheric Biennial Oscillations occur, with the implicit assumption that the oscillations are not just the result of random cycles of white noise.

[2]G. A. Meehl and J. M. Arblaster, “Relating the strength of the tropospheric biennial oscillation (TBO) to the phase of the Interdecadal Pacific Oscillation (IPO),” Geophysical Research Letters, vol. 39, no. 20, 2012.

[3]E. M. Rasmusson, X. Wang, and C. F. Ropelewski, “The biennial component of ENSO variability,” Journal of Marine Systems, vol. 1, no. 1, pp. 71–96, 1990.

[4]S.-R. Yeo and K.-Y. Kim, “Global warming, low-frequency variability, and biennial oscillation: an attempt to understand the physical mechanisms driving major ENSO events,” Climate Dynamics, vol. 43, no. 3–4, pp. 771–786, 2014.

As an alternate view, if the oscillations had an average value of 0, and equal probabilities above and below zero, and a requirement that a biennial oscillation would require a flip from + to - amplitude (or vice versa) then the 50% white noise likelihood would hold.

This is a nice word problem that everyone likely has an opinion on.

`This climate research group[1] claims that biennial oscillations are indistinguishable from white noise http://www.researchgate.net/publication/281899198_Tropospheric_Biennial_Oscillation_TBO_indistinguishable_from_white_noise [1]M. F. Stuecker, A. Timmermann, J. Yoon, and F. Jin, “Tropospheric Biennial Oscillation (TBO) indistinguishable from white noise,” Geophysical Research Letters, vol. 42, no. 18, pp. 7785–7791, 2015. They see the probability of detecting two-year oscillation periods in the fluctuations of white noise at 2/3. For discrete coin flipping, the probability has to be locked at 1/2, but since climate measures are continuous, the arithmetic of PDFs will weigh the results toward a flip in direction every year. I can see how this can occur : Say a value is at 0.8 in a range of 0 to 1 for one particular year. For white noise, the value can be anything so there is an 80% chance that it will be lower the next year. That change in direction from a high value to a low value (or vice versa) constitutes a biennial oscillation from their point-of-view, What they are trying to do is marginalize the idea of Meehl[2] and others[3,4] who claim that Tropospheric Biennial Oscillations occur, with the implicit assumption that the oscillations are not just the result of random cycles of white noise. [2]G. A. Meehl and J. M. Arblaster, “Relating the strength of the tropospheric biennial oscillation (TBO) to the phase of the Interdecadal Pacific Oscillation (IPO),” Geophysical Research Letters, vol. 39, no. 20, 2012. [3]E. M. Rasmusson, X. Wang, and C. F. Ropelewski, “The biennial component of ENSO variability,” Journal of Marine Systems, vol. 1, no. 1, pp. 71–96, 1990. [4]S.-R. Yeo and K.-Y. Kim, “Global warming, low-frequency variability, and biennial oscillation: an attempt to understand the physical mechanisms driving major ENSO events,” Climate Dynamics, vol. 43, no. 3–4, pp. 771–786, 2014. As an alternate view, if the oscillations had an average value of 0, and equal probabilities above and below zero, and a requirement that a biennial oscillation would require a flip from + to - amplitude (or vice versa) then the 50% white noise likelihood would hold. This is a nice word problem that everyone likely has an opinion on.`

I think Stuecker et al are right. They are using a criterion defined by other authors (Meehl etc) and rightfully criticising it as as biased.

They make a meal out of their analytical proof. If $X,Y,Z$ are three independent random variables from the same continuous distribution, then each ordering occurs with probability 1/6. Both $X \lt Z \lt Y$ and $Z \lt X \lt Y$ satisfy $X \lt Y \gt Z$. Both $Y \lt X \lt Z$ and $Y \lt Z \lt X$ satisfy $X \gt Y \lt Z$. Hence 2/3.

`I think Stuecker et al are right. They are using a criterion defined by other authors (Meehl etc) and rightfully criticising it as as biased. They make a meal out of their analytical proof. If $X,Y,Z$ are three independent random variables from the same continuous distribution, then each ordering occurs with probability 1/6. Both $X \lt Z \lt Y$ and $Z \lt X \lt Y$ satisfy $X \lt Y \gt Z$. Both $Y \lt X \lt Z$ and $Y \lt Z \lt X$ satisfy $X \gt Y \lt Z$. Hence 2/3.`