It looks like you're new here. If you want to get involved, click one of these buttons!

- All Categories 2.3K
- Chat 500
- Study Groups 19
- Petri Nets 9
- Epidemiology 4
- Leaf Modeling 1
- Review Sections 9
- MIT 2020: Programming with Categories 51
- MIT 2020: Lectures 20
- MIT 2020: Exercises 25
- MIT 2019: Applied Category Theory 339
- MIT 2019: Lectures 79
- MIT 2019: Exercises 149
- MIT 2019: Chat 50
- UCR ACT Seminar 4
- General 67
- Azimuth Code Project 110
- Statistical methods 4
- Drafts 2
- Math Syntax Demos 15
- Wiki - Latest Changes 3
- Strategy 113
- Azimuth Project 1.1K
- - Spam 1
- News and Information 147
- Azimuth Blog 149
- - Conventions and Policies 21
- - Questions 43
- Azimuth Wiki 713

Options

Hello John

SVR delta forecast of El Nino 3.4 Anomalies

Could you check and see if I took the right data, I copied it from your github address you had issued earlier:

https://raw.githubusercontent.com/johncarlosbaez/el-nino/master/R/nino3.4-anoms.txt

I could easily switch to another data.

I included all the algebra and math for SVR, I used someone else's professional mathematical writings to avoid my own.

Dara

## Comments

John if suddenly you do not hear from me, is because of power outage a new thing happening at Ireland due to severe winds due to weather changes recently. I will run to a hotel in a nearby town to have access.

`John if suddenly you do not hear from me, is because of power outage a new thing happening at Ireland due to severe winds due to weather changes recently. I will run to a hotel in a nearby town to have access.`

In this write up you can see how the actual curve-fit's accuracy does not match the forecast accuracy! I babbled earlier about this to Paul showing some amazing curve-fitting (some paper from another author). This is deceptive as I warned Paul, since the curve-fitting error is far less than forecast due to the time shift of 1.

So SVR's curve fitting is really great, but even that is not good enough for forecasts.

Therefore the next month forecast has 10% error with Max 0.84 out of 2.3 max of the original signal, so it is an iffy forecast.

But the further look-aheads into future are too inaccurate. For example 2nd month lookahead has error of 21%.

It seems the same range errors were uncovered from earlier work I did for you on NN, but I will return the numbers.

Dara

`In this write up you can see how the actual curve-fit's accuracy does not match the forecast accuracy! I babbled earlier about this to Paul showing some amazing curve-fitting (some paper from another author). This is deceptive as I warned Paul, since the curve-fitting error is far less than forecast due to the time shift of 1. So SVR's curve fitting is really great, but even that is not good enough for forecasts. Therefore the next month forecast has 10% error with Max 0.84 out of 2.3 max of the original signal, so it is an iffy forecast. But the further look-aheads into future are too inaccurate. For example 2nd month lookahead has error of 21%. It seems the same range errors were uncovered from earlier work I did for you on NN, but I will return the numbers. Dara`

John

The delta model which is a simple first-order difference equation could be replaced by higher order difference models, if you come up with some, I could code it for you. Maybe that is what is needed to increase the accuracy of the forecast.

`John The delta model which is a simple first-order difference equation could be replaced by higher order difference models, if you come up with some, I could code it for you. Maybe that is what is needed to increase the accuracy of the forecast.`

I added the Wavelet Transform Trend forecast which seems to be more accurate, it is in the code but I did not include in this report. If you need it I could issue it as well.

`I added the Wavelet Transform Trend forecast which seems to be more accurate, it is in the code but I did not include in this report. If you need it I could issue it as well.`

The plots represent 300 SVR forecasts for past 300 months of the data, each forecast uses 37 previous months from the present time. This is called

Backtesting. So the error analysis is carried out at each month and then averages issued.The global minimizer is Differential Evolution.

Note that the global minimizer solution to quadratic is not unique!

Dara

`The plots represent 300 SVR forecasts for past 300 months of the data, each forecast uses 37 previous months from the present time. This is called **Backtesting**. So the error analysis is carried out at each month and then averages issued. The global minimizer is Differential Evolution. Note that the global minimizer solution to quadratic is not unique! Dara`

John

If you look at the math write up in the paper, you could easily take these ideas for SVR and the delta difference I added and propose large number of new such algorithms, using a global minimizer.

I suspect the folks who do this research are actually not good mathematicians as you are, so the research lacks! But if you are willing to spend the energy, you could advance this field easily into a new domain

Dara

`John If you look at the math write up in the paper, you could easily take these ideas for SVR and the delta difference I added and propose large number of new such algorithms, using a global minimizer. I suspect the folks who do this research are actually not good mathematicians as you are, so the research lacks! But if you are willing to spend the energy, you could advance this field easily into a new domain Dara`

Dara, amazing the detail that you can achieve with those curves.

`Dara, amazing the detail that you can achieve with those curves.`

Paul scroll to the bottom of the write up see the kernel function, it is a wavelet.

Look at the quadratic function which serves as the interpolation. See the wavelet term.

I was proposing that we plugin those wavelets into your diff eq to localize the fit vs. the global fit you were doing.

I also want to re design the SVR, hopefully with John's cooperation, so in the equation that minimizer minimizes, actually solve a differential equation which has your theoretical model concepts e.g. sloshing or wind.

Dara

`Paul scroll to the bottom of the write up see the kernel function, it is a wavelet. Look at the quadratic function which serves as the interpolation. See the wavelet term. I was proposing that we plugin those wavelets into your diff eq to localize the fit vs. the global fit you were doing. I also want to re design the SVR, hopefully with John's cooperation, so in the equation that minimizer minimizes, actually solve a differential equation which has your theoretical model concepts e.g. sloshing or wind. Dara`

Dara, My concern about localizing the fit versus making a fit that spans 100+ years is that a localized fit will tend to follow spurious noise excursions. For example, in terms of the SOI, the curve can fit either Darwin or Tahiti, or the difference. We don't know which of these is the true value. But by fitting over a longer range, the spurious nature gets averaged out.

It would be different if there was a measure that exposed the true ENSO characteristic, but alas, I don't think that such a beast exists. Every measure is but a representation of what is actually happening in the ocean.

By that token, extending to volumetric measures is probably the best bet and getting the computational horsepower to harness that problem is what you can do best.

But this is just my gut feel, having battled the data for several months now.

`Dara, My concern about localizing the fit versus making a fit that spans 100+ years is that a localized fit will tend to follow spurious noise excursions. For example, in terms of the SOI, the curve can fit either Darwin or Tahiti, or the difference. We don't know which of these is the true value. But by fitting over a longer range, the spurious nature gets averaged out. It would be different if there was a measure that exposed the true ENSO characteristic, but alas, I don't think that such a beast exists. Every measure is but a representation of what is actually happening in the ocean. By that token, extending to volumetric measures is probably the best bet and getting the computational horsepower to harness that problem is what you can do best. But this is just my gut feel, having battled the data for several months now.`