Options

Information-non increase: a kolmogorov complexity theorem

Kolmogorov Complexity

Theorem

information non-increase K(f(x))≤K(x)+K(f) for computable functions f

If you run a computation function e.g. software algorithm on data, you cannot increase its information content by much except the length of the program itself.

Therefore if you are making a computation forecast based upon finite data, your forecast has certain amount of accuracy which is related to total amount information encapsulated into the data. If you run a computational functions on that data e.g. link strength or averaging you cannot increase the amount of information in that data by much i.e. almost nothing.

Therefore manipulation of the signal by computational functions will not increase its forecast accuracy by much, since it will not increase its amount of information.

What you could do is to add new data, which adds new information, therefore the amount of information is increased, possibly then you could make a more accurate forecast.

Comments

  • 1.

    In other words, you don't get something from nothing --- aka there is no such thing as a free lunch.

    Comment Source:In other words, you don't get something from nothing --- *aka* there is no such thing as a free lunch.
  • 2.

    Correct Paul, so if we see that we run several machine learning algorithms on El Nino index, as I did and they all give similar accuracy, we conclude that we need to add additional data to have additional information content for better forecast, keep averaging a number of columns and rows for the data will not yield any addition to the information content, since the algorithms e.g. averaging is tiny in length.

    Therefore we need to add additional data, but as you said earlier something relevant which could help in forecast.

    For me that is an important consideration for John to help out to see what additional data needs to be added?

    Comment Source:Correct Paul, so if we see that we run several machine learning algorithms on El Nino index, as I did and they all give similar accuracy, we conclude that we need to add additional data to have additional information content for better forecast, keep averaging a number of columns and rows for the data will not yield any addition to the information content, since the algorithms e.g. averaging is tiny in length. Therefore we need to add additional data, but as you said earlier something relevant which could help in forecast. For me that is an important consideration for John to help out to see what additional data needs to be added?
Sign In or Register to comment.