It looks like you're new here. If you want to get involved, click one of these buttons!

- All Categories 2.4K
- Chat 505
- Study Groups 21
- Petri Nets 9
- Epidemiology 4
- Leaf Modeling 2
- Review Sections 9
- MIT 2020: Programming with Categories 51
- MIT 2020: Lectures 20
- MIT 2020: Exercises 25
- Baez ACT 2019: Online Course 339
- Baez ACT 2019: Lectures 79
- Baez ACT 2019: Exercises 149
- Baez ACT 2019: Chat 50
- UCR ACT Seminar 4
- General 75
- Azimuth Code Project 111
- Statistical methods 4
- Drafts 10
- Math Syntax Demos 15
- Wiki - Latest Changes 3
- Strategy 113
- Azimuth Project 1.1K
- - Spam 1
- News and Information 148
- Azimuth Blog 149
- - Conventions and Policies 21
- - Questions 43
- Azimuth Wiki 719

Options

k-NN Delta-Forecast for El Nino 3.4 Anomalies

This is the simplest algorithm for Machine Learning forecasts, in particular it looks for signal self-similarity and averages out the out from each similar case called Nearest-neighbour.

Similar error of 9% to SVR and NN is found.

All known and popular distance functions were tried, but the most accurate forecasts were obtained by non-Euclidean metrics.

## Comments

I had expected lesser accuracy for k-NN but since the signal is mostly self-similar i.e. periodic then k-NN has performed well.

I love k-NN since it allows us to deal with metric spaces and maps between them, so it is a novel modern approach to understanding the self-similarity of almost any arbitrary data.

It is almost incredibly fast which makes it suitable for a quick analysis of the data for very large datasets

`I had expected lesser accuracy for k-NN but since the signal is mostly self-similar i.e. periodic then k-NN has performed well. I love k-NN since it allows us to deal with metric spaces and maps between them, so it is a novel modern approach to understanding the self-similarity of almost any arbitrary data. It is almost incredibly fast which makes it suitable for a quick analysis of the data for very large datasets`

I forgot, k-NN could easily be parallelized for very large data sets

`I forgot, k-NN could easily be parallelized for very large data sets`

Dara, It looks as if there is a lot of compensating corrections going on in that algorithm. I can sense that because the forecast always seems to be lagging, sometimes ever-so-slightly, from the actual curve. And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly.

So you are saying the information content is limited to just a few near-neighbors, and that defines how far you can forecast into the future?

I am still subscribing to the saying that you can't get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators, such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast?

`Dara, It looks as if there is a lot of compensating corrections going on in that algorithm. I can sense that because the forecast always seems to be lagging, sometimes ever-so-slightly, from the actual curve. And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly. So you are saying the information content is limited to just a few near-neighbors, and that defines how far you can forecast into the future? I am still subscribing to the saying that you can't get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators, such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast?`

That is adaptive learning! you do that when you dance or play music in a group or play a sport.

Pretty much similar, but of course the devil is in the details.

I like to propose to John and you and other programmers here, to use k-NN more often, it is a simple fast and accurate enough learning algorithm for us to get a view of the playing field for a data

Dara

`>And in places that it makes a sudden lurch away from the actual, it gets brought back in-line quickly. That is adaptive learning! you do that when you dance or play music in a group or play a sport. >such as embodied by Kalman filters. These do work well for short-term forecasting, such as navigation control systems, but how far in advance can they really forecast? Pretty much similar, but of course the devil is in the details. I like to propose to John and you and other programmers here, to use k-NN more often, it is a simple fast and accurate enough learning algorithm for us to get a view of the playing field for a data Dara`

Actually they are nothing but non-linear adaptive curve fitters i.e. each point of time a different estimators! So they are localized curve fitters as opposed to the global ones like Splines.

More and more we are discovering how simple they are, that is why I want to finish the parallelized Differential Evolution for John to start designing new such adaptive algorithms, as you noted earlier mixed with differential equations.

`>I am still subscribing to the saying that you can’t get something from nothing. So, without understanding anything about the underlying algorithm, it appears that these curves are very similar to continuously updated estimators Actually they are nothing but non-linear adaptive curve fitters i.e. each point of time a different estimators! So they are localized curve fitters as opposed to the global ones like Splines. More and more we are discovering how simple they are, that is why I want to finish the parallelized Differential Evolution for John to start designing new such adaptive algorithms, as you noted earlier mixed with differential equations.`

Dara,

OK, now I understand much better. We try to get the best of both worlds -- a simplifying physics-based model together with an adaptive algorithm for estimating.

It may make the whole greater than the sum of the parts. One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal. This might just be the ticket, but then we have to explain the approach effectively to get buy-in.

`Dara, OK, now I understand much better. We try to get the best of both worlds -- a simplifying physics-based model together with an adaptive algorithm for estimating. It may make the whole greater than the sum of the parts. One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal. This might just be the ticket, but then we have to explain the approach effectively to get buy-in.`

ROGER!

However with one addition: to decompose the dynamical system into synchronous and a-synchronous decompositions and model the a-synchronous part with Petri Nets and the synchronous parts with Machine Learning.

Example: Use the k-NN in some normed vector space and see how many Nearest neighbours there are at a time, do a k-NN regression get a number which then has a threshold that places a marker/token into a place for the Petri Net which might fire a-synchronously.

The part I did not know for years was how to make these sorts of ideas fast, now I learned past few years how to design large clouds of multi-code CPUs and GPUs to highly parallelize.

`>a simplifying physics-based model together with an adaptive algorithm for estimating. ROGER! >One side tries to improve the understanding of the fundamental physical process while the other side tries to isolate the intrinsic signal. However with one addition: to decompose the dynamical system into synchronous and a-synchronous decompositions and model the a-synchronous part with Petri Nets and the synchronous parts with Machine Learning. Example: Use the k-NN in some normed vector space and see how many Nearest neighbours there are at a time, do a k-NN regression get a number which then has a threshold that places a marker/token into a place for the Petri Net which might fire a-synchronously. The part I did not know for years was how to make these sorts of ideas fast, now I learned past few years how to design large clouds of multi-code CPUs and GPUs to highly parallelize.`

No links, got an improvement:

k-NN regression El Nino 3.4 Anomalies, no links

This is the interlaced i.e. anom1, c1, anom2, c2, anom3, c3 ... where c could be any real number:

[k-NN regression El Nino 3.4 Anomalies interlaced

One is (Link, Anomalies) interlaced and the link data is Daniel's. Another is (1, Anomalies) Anomalies interlaced with constant 1. Another is (0, Anomalies) Anomalies interlaced with constant 0.

`No links, got an improvement: [k-NN regression El Nino 3.4 Anomalies, no links](http://files.lossofgenerality.com/knn1_nino34_anom_IMPROVED.pdf) This is the interlaced i.e. anom1, c1, anom2, c2, anom3, c3 ... where c could be any real number: [[k-NN regression El Nino 3.4 Anomalies interlaced](http://files.lossofgenerality.com/knn2_nino34_anom_link.pdf) One is (Link, Anomalies) interlaced and the link data is Daniel's. Another is (1, Anomalies) Anomalies interlaced with constant 1. Another is (0, Anomalies) Anomalies interlaced with constant 0.`

For the links interlaced did a forecast without using the delta method and the accuracy was much lower.

`For the links interlaced did a forecast without using the delta method and the accuracy was much lower.`