I doubt that noise is much of an issue -- we are considering an ocean basin with a significant inertia associated with the thermocline. Nothing has been known to disturb the cycles of ENSO, not even volcanic events. Contrast that to atmospheric cycles, where the lower density and thus lower inertia will make it more sensitive to disturbances, which is likely why the QBO shows more noisy anomalies.

You said:

> "Its becoming clear that ENSO data noise causes overfitting of virtually all numeric models to date, and incomplete model identification similarly causes pervasive underfitting."

Let's place some context into the fitting issue. Recall that *only two* primary lunar cycles are required to match the ENSO time-series, subject to the non-linear transformation derived from solving Laplace's Tidal Equations. Here, *F(t)* is the tidal forcing

$$ \sin (A F(t)) $$

In contrast, conventional tidal analysis is just the linear transformation

$$ A F(t) $$

The difference between the two in terms of structural sensitivity is like the diff between night and day. For conventional tidal analysis, changing the forcing scale factor *A* will maintain the integrity of the underling pattern. So one can add additional tidal harmonics and understand precisely what it will do to the result.

$$ dA \cdot F(t) + A \cdot dF(t) $$

Unfortunately that is not the case with the non-linear LTE. A differential applied as such

$$ \cos (A F(t)) ( dA \cdot F(t) + A \cdot dF(t) ) $$

obviously does a non-intuitive squirrelly scaling, almost 90 degrees to what one would expect to a linear tidal response -- since the cosine is orthogonal to the sine. The outcome of this is that the iteration to an optimal fit takes time, and even a slight delta change can wreak havoc on a good fit.

So both under-fitting and over-fitting will be sub-optimal and only a precise fit will do the job properly. There are a number of non-chaotic physical processes that are analogous to "threading the needle" and can only be analyzed by nailing the pattern exactly right. Not surprisingly the analysis of [liquid sloshing in a tank](https://asmedigitalcollection.asme.org/fluidsengineering/article/137/9/090801/472122/Recent-Advances-in-Physics-of-Fluid-Parametric) has the same exacting requirements.

This essentially explains where I spend most of my time in the analysis, calibrating and cross-validating all the forces required to thread the needle.

---

BTW, what's crucial about the finding is that there is a new initiative in climate modeling (https://clima.caltech.edu/) that plans to use alternative approaches such as machine learning to discover climate and ocean patterns.

> "CLIMATE MACHINE -- We are developing the first Earth system model that automatically learns from diverse data sources. Our model will exploit advances in **machine learning** and data assimilation to learn from observations and from data generated on demand in targeted high-resolution simulations, for example, of clouds or **ocean turbulence**. This will allow us to reduce and quantify uncertainties in climate predictions."

This project has significant government and private funding so I have no doubt that some deep learning algorithm will discover the same underlying pattern I have, but instead of by a careful application of physics it will do it via brute force means. Then watch what happens when they try to reverse engineer the ML results.

You said:

> "Its becoming clear that ENSO data noise causes overfitting of virtually all numeric models to date, and incomplete model identification similarly causes pervasive underfitting."

Let's place some context into the fitting issue. Recall that *only two* primary lunar cycles are required to match the ENSO time-series, subject to the non-linear transformation derived from solving Laplace's Tidal Equations. Here, *F(t)* is the tidal forcing

$$ \sin (A F(t)) $$

In contrast, conventional tidal analysis is just the linear transformation

$$ A F(t) $$

The difference between the two in terms of structural sensitivity is like the diff between night and day. For conventional tidal analysis, changing the forcing scale factor *A* will maintain the integrity of the underling pattern. So one can add additional tidal harmonics and understand precisely what it will do to the result.

$$ dA \cdot F(t) + A \cdot dF(t) $$

Unfortunately that is not the case with the non-linear LTE. A differential applied as such

$$ \cos (A F(t)) ( dA \cdot F(t) + A \cdot dF(t) ) $$

obviously does a non-intuitive squirrelly scaling, almost 90 degrees to what one would expect to a linear tidal response -- since the cosine is orthogonal to the sine. The outcome of this is that the iteration to an optimal fit takes time, and even a slight delta change can wreak havoc on a good fit.

So both under-fitting and over-fitting will be sub-optimal and only a precise fit will do the job properly. There are a number of non-chaotic physical processes that are analogous to "threading the needle" and can only be analyzed by nailing the pattern exactly right. Not surprisingly the analysis of [liquid sloshing in a tank](https://asmedigitalcollection.asme.org/fluidsengineering/article/137/9/090801/472122/Recent-Advances-in-Physics-of-Fluid-Parametric) has the same exacting requirements.

This essentially explains where I spend most of my time in the analysis, calibrating and cross-validating all the forces required to thread the needle.

---

BTW, what's crucial about the finding is that there is a new initiative in climate modeling (https://clima.caltech.edu/) that plans to use alternative approaches such as machine learning to discover climate and ocean patterns.

> "CLIMATE MACHINE -- We are developing the first Earth system model that automatically learns from diverse data sources. Our model will exploit advances in **machine learning** and data assimilation to learn from observations and from data generated on demand in targeted high-resolution simulations, for example, of clouds or **ocean turbulence**. This will allow us to reduce and quantify uncertainties in climate predictions."

This project has significant government and private funding so I have no doubt that some deep learning algorithm will discover the same underlying pattern I have, but instead of by a careful application of physics it will do it via brute force means. Then watch what happens when they try to reverse engineer the ML results.