#### Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Options

# k-NN regression:el nino 3.4 anomalies index

k-NN Delta-Forecast for El Nino 3.4 Anomalies

This is the simplest algorithm for Machine Learning forecasts, in particular it looks for signal self-similarity and averages out the out from each similar case called Nearest-neighbour.

Similar error of 9% to SVR and NN is found.

All known and popular distance functions were tried, but the most accurate forecasts were obtained by non-Euclidean metrics.

• Options
1.

I had expected lesser accuracy for k-NN but since the signal is mostly self-similar i.e. periodic then k-NN has performed well.

I love k-NN since it allows us to deal with metric spaces and maps between them, so it is a novel modern approach to understanding the self-similarity of almost any arbitrary data.

It is almost incredibly fast which makes it suitable for a quick analysis of the data for very large datasets

Comment Source:I had expected lesser accuracy for k-NN but since the signal is mostly self-similar i.e. periodic then k-NN has performed well. I love k-NN since it allows us to deal with metric spaces and maps between them, so it is a novel modern approach to understanding the self-similarity of almost any arbitrary data. It is almost incredibly fast which makes it suitable for a quick analysis of the data for very large datasets
• Options
2.

I forgot, k-NN could easily be parallelized for very large data sets

Comment Source:I forgot, k-NN could easily be parallelized for very large data sets
• Options
3.

Dara, It looks as if there is a lot of compensating corrections going on in that algorithm. I can sense that because the forecast always seems to be lagging, sometimes ever-so-slightly, from the actual curve. And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly.

So you are saying the information content is limited to just a few near-neighbors, and that defines how far you can forecast into the future?

I am still subscribing to the saying that you can't get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators, such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast?

Comment Source:Dara, It looks as if there is a lot of compensating corrections going on in that algorithm. I can sense that because the forecast always seems to be lagging, sometimes ever-so-slightly, from the actual curve. And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly. So you are saying the information content is limited to just a few near-neighbors, and that defines how far you can forecast into the future? I am still subscribing to the saying that you can't get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators, such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast?
• Options
4.

And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly.

That is adaptive learning! you do that when you dance or play music in a group or play a sport.

such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast?

Pretty much similar, but of course the devil is in the details.

I like to propose to John and you and other programmers here, to use k-NN more often, it is a simple fast and accurate enough learning algorithm for us to get a view of the playing field for a data

Dara

Comment Source:>And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly. That is adaptive learning! you do that when you dance or play music in a group or play a sport. >such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast? Pretty much similar, but of course the devil is in the details. I like to propose to John and you and other programmers here, to use k-NN more often, it is a simple fast and accurate enough learning algorithm for us to get a view of the playing field for a data Dara
• Options
5.

I am still subscribing to the saying that you can’t get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators

Actually they are nothing but non-linear adaptive curve fitters i.e. each point of time a different estimators! So they are localized curve fitters as opposed to the global ones like Splines.

More and more we are discovering how simple they are, that is why I want to finish the parallelized Differential Evolution for John to start designing new such adaptive algorithms, as you noted earlier mixed with differential equations.

Comment Source:>I am still subscribing to the saying that you can’t get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators Actually they are nothing but non-linear adaptive curve fitters i.e. each point of time a different estimators! So they are localized curve fitters as opposed to the global ones like Splines. More and more we are discovering how simple they are, that is why I want to finish the parallelized Differential Evolution for John to start designing new such adaptive algorithms, as you noted earlier mixed with differential equations.
• Options
6.

Dara,
OK, now I understand much better. We try to get the best of both worlds -- a simplifying physics-based model together with an adaptive algorithm for estimating.

It may make the whole greater than the sum of the parts. One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal. This might just be the ticket, but then we have to explain the approach effectively to get buy-in.

Comment Source:Dara, OK, now I understand much better. We try to get the best of both worlds -- a simplifying physics-based model together with an adaptive algorithm for estimating. It may make the whole greater than the sum of the parts. One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal. This might just be the ticket, but then we have to explain the approach effectively to get buy-in.
• Options
7.

a simplifying physics-based model together with an adaptive algorithm for estimating.

ROGER!

One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal.

However with one addition: to decompose the dynamical system into synchronous and a-synchronous decompositions and model the a-synchronous part with Petri Nets and the synchronous parts with Machine Learning.

Example: Use the k-NN in some normed vector space and see how many Nearest neighbours there are at a time, do a k-NN regression get a number which then has a threshold that places a marker/token into a place for the Petri Net which might fire a-synchronously.

The part I did not know for years was how to make these sorts of ideas fast, now I learned past few years how to design large clouds of multi-code CPUs and GPUs to highly parallelize.

Comment Source:>a simplifying physics-based model together with an adaptive algorithm for estimating. ROGER! >One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal. However with one addition: to decompose the dynamical system into synchronous and a-synchronous decompositions and model the a-synchronous part with Petri Nets and the synchronous parts with Machine Learning. Example: Use the k-NN in some normed vector space and see how many Nearest neighbours there are at a time, do a k-NN regression get a number which then has a threshold that places a marker/token into a place for the Petri Net which might fire a-synchronously. The part I did not know for years was how to make these sorts of ideas fast, now I learned past few years how to design large clouds of multi-code CPUs and GPUs to highly parallelize.
• Options
8.

No links, got an improvement:

k-NN regression El Nino 3.4 Anomalies, no links

This is the interlaced i.e. anom1, c1, anom2, c2, anom3, c3 ... where c could be any real number:

One is (Link, Anomalies) interlaced and the link data is Daniel's. Another is (1, Anomalies) Anomalies interlaced with constant 1. Another is (0, Anomalies) Anomalies interlaced with constant 0.

Comment Source:No links, got an improvement: [k-NN regression El Nino 3.4 Anomalies, no links](http://files.lossofgenerality.com/knn1_nino34_anom_IMPROVED.pdf) This is the interlaced i.e. anom1, c1, anom2, c2, anom3, c3 ... where c could be any real number: [[k-NN regression El Nino 3.4 Anomalies interlaced](http://files.lossofgenerality.com/knn2_nino34_anom_link.pdf) One is (Link, Anomalies) interlaced and the link data is Daniel's. Another is (1, Anomalies) Anomalies interlaced with constant 1. Another is (0, Anomalies) Anomalies interlaced with constant 0.
• Options
9.

For the links interlaced did a forecast without using the delta method and the accuracy was much lower.

Comment Source:For the links interlaced did a forecast without using the delta method and the accuracy was much lower.