Options

SVR: el nino 3.4 anomalies index

Hello John

SVR delta forecast of El Nino 3.4 Anomalies

Could you check and see if I took the right data, I copied it from your github address you had issued earlier:

https://raw.githubusercontent.com/johncarlosbaez/el-nino/master/R/nino3.4-anoms.txt

I could easily switch to another data.

I included all the algebra and math for SVR, I used someone else's professional mathematical writings to avoid my own.

Dara

Comments

  • 1.

    John if suddenly you do not hear from me, is because of power outage a new thing happening at Ireland due to severe winds due to weather changes recently. I will run to a hotel in a nearby town to have access.

    Comment Source:John if suddenly you do not hear from me, is because of power outage a new thing happening at Ireland due to severe winds due to weather changes recently. I will run to a hotel in a nearby town to have access.
  • 2.
    edited November 2014

    In this write up you can see how the actual curve-fit's accuracy does not match the forecast accuracy! I babbled earlier about this to Paul showing some amazing curve-fitting (some paper from another author). This is deceptive as I warned Paul, since the curve-fitting error is far less than forecast due to the time shift of 1.

    So SVR's curve fitting is really great, but even that is not good enough for forecasts.

    Therefore the next month forecast has 10% error with Max 0.84 out of 2.3 max of the original signal, so it is an iffy forecast.

    But the further look-aheads into future are too inaccurate. For example 2nd month lookahead has error of 21%.

    It seems the same range errors were uncovered from earlier work I did for you on NN, but I will return the numbers.

    Dara

    Comment Source:In this write up you can see how the actual curve-fit's accuracy does not match the forecast accuracy! I babbled earlier about this to Paul showing some amazing curve-fitting (some paper from another author). This is deceptive as I warned Paul, since the curve-fitting error is far less than forecast due to the time shift of 1. So SVR's curve fitting is really great, but even that is not good enough for forecasts. Therefore the next month forecast has 10% error with Max 0.84 out of 2.3 max of the original signal, so it is an iffy forecast. But the further look-aheads into future are too inaccurate. For example 2nd month lookahead has error of 21%. It seems the same range errors were uncovered from earlier work I did for you on NN, but I will return the numbers. Dara
  • 3.

    John

    The delta model which is a simple first-order difference equation could be replaced by higher order difference models, if you come up with some, I could code it for you. Maybe that is what is needed to increase the accuracy of the forecast.

    Comment Source:John The delta model which is a simple first-order difference equation could be replaced by higher order difference models, if you come up with some, I could code it for you. Maybe that is what is needed to increase the accuracy of the forecast.
  • 4.

    I added the Wavelet Transform Trend forecast which seems to be more accurate, it is in the code but I did not include in this report. If you need it I could issue it as well.

    Comment Source:I added the Wavelet Transform Trend forecast which seems to be more accurate, it is in the code but I did not include in this report. If you need it I could issue it as well.
  • 5.

    The plots represent 300 SVR forecasts for past 300 months of the data, each forecast uses 37 previous months from the present time. This is called Backtesting. So the error analysis is carried out at each month and then averages issued.

    The global minimizer is Differential Evolution.

    Note that the global minimizer solution to quadratic is not unique!

    Dara

    Comment Source:The plots represent 300 SVR forecasts for past 300 months of the data, each forecast uses 37 previous months from the present time. This is called **Backtesting**. So the error analysis is carried out at each month and then averages issued. The global minimizer is Differential Evolution. Note that the global minimizer solution to quadratic is not unique! Dara
  • 6.

    John

    If you look at the math write up in the paper, you could easily take these ideas for SVR and the delta difference I added and propose large number of new such algorithms, using a global minimizer.

    I suspect the folks who do this research are actually not good mathematicians as you are, so the research lacks! But if you are willing to spend the energy, you could advance this field easily into a new domain

    Dara

    Comment Source:John If you look at the math write up in the paper, you could easily take these ideas for SVR and the delta difference I added and propose large number of new such algorithms, using a global minimizer. I suspect the folks who do this research are actually not good mathematicians as you are, so the research lacks! But if you are willing to spend the energy, you could advance this field easily into a new domain Dara
  • 7.

    Dara, amazing the detail that you can achieve with those curves.

    Comment Source:Dara, amazing the detail that you can achieve with those curves.
  • 8.

    Paul scroll to the bottom of the write up see the kernel function, it is a wavelet.

    Look at the quadratic function which serves as the interpolation. See the wavelet term.

    I was proposing that we plugin those wavelets into your diff eq to localize the fit vs. the global fit you were doing.

    I also want to re design the SVR, hopefully with John's cooperation, so in the equation that minimizer minimizes, actually solve a differential equation which has your theoretical model concepts e.g. sloshing or wind.

    Dara

    Comment Source:Paul scroll to the bottom of the write up see the kernel function, it is a wavelet. Look at the quadratic function which serves as the interpolation. See the wavelet term. I was proposing that we plugin those wavelets into your diff eq to localize the fit vs. the global fit you were doing. I also want to re design the SVR, hopefully with John's cooperation, so in the equation that minimizer minimizes, actually solve a differential equation which has your theoretical model concepts e.g. sloshing or wind. Dara
  • 9.

    Dara, My concern about localizing the fit versus making a fit that spans 100+ years is that a localized fit will tend to follow spurious noise excursions. For example, in terms of the SOI, the curve can fit either Darwin or Tahiti, or the difference. We don't know which of these is the true value. But by fitting over a longer range, the spurious nature gets averaged out.

    It would be different if there was a measure that exposed the true ENSO characteristic, but alas, I don't think that such a beast exists. Every measure is but a representation of what is actually happening in the ocean.

    By that token, extending to volumetric measures is probably the best bet and getting the computational horsepower to harness that problem is what you can do best.

    But this is just my gut feel, having battled the data for several months now.

    Comment Source:Dara, My concern about localizing the fit versus making a fit that spans 100+ years is that a localized fit will tend to follow spurious noise excursions. For example, in terms of the SOI, the curve can fit either Darwin or Tahiti, or the difference. We don't know which of these is the true value. But by fitting over a longer range, the spurious nature gets averaged out. It would be different if there was a measure that exposed the true ENSO characteristic, but alas, I don't think that such a beast exists. Every measure is but a representation of what is actually happening in the ocean. By that token, extending to volumetric measures is probably the best bet and getting the computational horsepower to harness that problem is what you can do best. But this is just my gut feel, having battled the data for several months now.
Sign In or Register to comment.