It looks like you're new here. If you want to get involved, click one of these buttons!

- All Categories 2.2K
- Applied Category Theory Course 352
- Applied Category Theory Seminar 4
- Exercises 149
- Discussion Groups 49
- How to Use MathJax 15
- Chat 479
- Azimuth Code Project 108
- News and Information 145
- Azimuth Blog 149
- Azimuth Forum 29
- Azimuth Project 189
- - Strategy 108
- - Conventions and Policies 21
- - Questions 43
- Azimuth Wiki 711
- - Latest Changes 701
- - - Action 14
- - - Biodiversity 8
- - - Books 2
- - - Carbon 9
- - - Computational methods 38
- - - Climate 53
- - - Earth science 23
- - - Ecology 43
- - - Energy 29
- - - Experiments 30
- - - Geoengineering 0
- - - Mathematical methods 69
- - - Meta 9
- - - Methodology 16
- - - Natural resources 7
- - - Oceans 4
- - - Organizations 34
- - - People 6
- - - Publishing 4
- - - Reports 3
- - - Software 21
- - - Statistical methods 2
- - - Sustainability 4
- - - Things to do 2
- - - Visualisation 1
- General 39

Options

k-NN Delta-Forecast for El Nino 3.4 Anomalies

This is the simplest algorithm for Machine Learning forecasts, in particular it looks for signal self-similarity and averages out the out from each similar case called Nearest-neighbour.

Similar error of 9% to SVR and NN is found.

All known and popular distance functions were tried, but the most accurate forecasts were obtained by non-Euclidean metrics.

## Comments

I had expected lesser accuracy for k-NN but since the signal is mostly self-similar i.e. periodic then k-NN has performed well.

I love k-NN since it allows us to deal with metric spaces and maps between them, so it is a novel modern approach to understanding the self-similarity of almost any arbitrary data.

It is almost incredibly fast which makes it suitable for a quick analysis of the data for very large datasets

`I had expected lesser accuracy for k-NN but since the signal is mostly self-similar i.e. periodic then k-NN has performed well. I love k-NN since it allows us to deal with metric spaces and maps between them, so it is a novel modern approach to understanding the self-similarity of almost any arbitrary data. It is almost incredibly fast which makes it suitable for a quick analysis of the data for very large datasets`

I forgot, k-NN could easily be parallelized for very large data sets

`I forgot, k-NN could easily be parallelized for very large data sets`

Dara, It looks as if there is a lot of compensating corrections going on in that algorithm. I can sense that because the forecast always seems to be lagging, sometimes ever-so-slightly, from the actual curve. And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly.

So you are saying the information content is limited to just a few near-neighbors, and that defines how far you can forecast into the future?

I am still subscribing to the saying that you can't get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators, such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast?

`Dara, It looks as if there is a lot of compensating corrections going on in that algorithm. I can sense that because the forecast always seems to be lagging, sometimes ever-so-slightly, from the actual curve. And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly. So you are saying the information content is limited to just a few near-neighbors, and that defines how far you can forecast into the future? I am still subscribing to the saying that you can't get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators, such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast?`

That is adaptive learning! you do that when you dance or play music in a group or play a sport.

Pretty much similar, but of course the devil is in the details.

I like to propose to John and you and other programmers here, to use k-NN more often, it is a simple fast and accurate enough learning algorithm for us to get a view of the playing field for a data

Dara

`>And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly. That is adaptive learning! you do that when you dance or play music in a group or play a sport. >such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast? Pretty much similar, but of course the devil is in the details. I like to propose to John and you and other programmers here, to use k-NN more often, it is a simple fast and accurate enough learning algorithm for us to get a view of the playing field for a data Dara`

Actually they are nothing but non-linear adaptive curve fitters i.e. each point of time a different estimators! So they are localized curve fitters as opposed to the global ones like Splines.

More and more we are discovering how simple they are, that is why I want to finish the parallelized Differential Evolution for John to start designing new such adaptive algorithms, as you noted earlier mixed with differential equations.

`>I am still subscribing to the saying that you can’t get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators Actually they are nothing but non-linear adaptive curve fitters i.e. each point of time a different estimators! So they are localized curve fitters as opposed to the global ones like Splines. More and more we are discovering how simple they are, that is why I want to finish the parallelized Differential Evolution for John to start designing new such adaptive algorithms, as you noted earlier mixed with differential equations.`

Dara,

OK, now I understand much better. We try to get the best of both worlds -- a simplifying physics-based model together with an adaptive algorithm for estimating.

It may make the whole greater than the sum of the parts. One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal. This might just be the ticket, but then we have to explain the approach effectively to get buy-in.

`Dara, OK, now I understand much better. We try to get the best of both worlds -- a simplifying physics-based model together with an adaptive algorithm for estimating. It may make the whole greater than the sum of the parts. One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal. This might just be the ticket, but then we have to explain the approach effectively to get buy-in.`

ROGER!

However with one addition: to decompose the dynamical system into synchronous and a-synchronous decompositions and model the a-synchronous part with Petri Nets and the synchronous parts with Machine Learning.

Example: Use the k-NN in some normed vector space and see how many Nearest neighbours there are at a time, do a k-NN regression get a number which then has a threshold that places a marker/token into a place for the Petri Net which might fire a-synchronously.

The part I did not know for years was how to make these sorts of ideas fast, now I learned past few years how to design large clouds of multi-code CPUs and GPUs to highly parallelize.

`>a simplifying physics-based model together with an adaptive algorithm for estimating. ROGER! >One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal. However with one addition: to decompose the dynamical system into synchronous and a-synchronous decompositions and model the a-synchronous part with Petri Nets and the synchronous parts with Machine Learning. Example: Use the k-NN in some normed vector space and see how many Nearest neighbours there are at a time, do a k-NN regression get a number which then has a threshold that places a marker/token into a place for the Petri Net which might fire a-synchronously. The part I did not know for years was how to make these sorts of ideas fast, now I learned past few years how to design large clouds of multi-code CPUs and GPUs to highly parallelize.`

No links, got an improvement:

k-NN regression El Nino 3.4 Anomalies, no links

This is the interlaced i.e. anom1, c1, anom2, c2, anom3, c3 ... where c could be any real number:

[k-NN regression El Nino 3.4 Anomalies interlaced

One is (Link, Anomalies) interlaced and the link data is Daniel's. Another is (1, Anomalies) Anomalies interlaced with constant 1. Another is (0, Anomalies) Anomalies interlaced with constant 0.

`No links, got an improvement: [k-NN regression El Nino 3.4 Anomalies, no links](http://files.lossofgenerality.com/knn1_nino34_anom_IMPROVED.pdf) This is the interlaced i.e. anom1, c1, anom2, c2, anom3, c3 ... where c could be any real number: [[k-NN regression El Nino 3.4 Anomalies interlaced](http://files.lossofgenerality.com/knn2_nino34_anom_link.pdf) One is (Link, Anomalies) interlaced and the link data is Daniel's. Another is (1, Anomalies) Anomalies interlaced with constant 1. Another is (0, Anomalies) Anomalies interlaced with constant 0.`

For the links interlaced did a forecast without using the delta method and the accuracy was much lower.

`For the links interlaced did a forecast without using the delta method and the accuracy was much lower.`