I would be curious to learn more about Fisher information for the following reason.
Here in Lithuania, Šarūnas Raudys won a national science award for his work on classifiers in statistics and neural networks. He wrote a book, Statistical and Neural Classifiers: An Integrated Approach to Design
He describes the following hierarchy of classifiers:
- the Euclidean distance classifier;
- the standard Fisher linear discriminant function (DF);
- the Fisher linear DF with pseudo-inversion of the covariance matrix;
- regularized linear discriminant analysis;
- the generalized Fisher DF;
- the minimum empirical error classifier;
- the maximum margin classifier.
His main conclusion is that it is very important, in training neural networks, that they not overlearn. Which is to say, there always needs to be some noise in the training so that they stay open to new types of learning, which can be more sophisticated, as per this hierarchy. If they overlearn, then they will be stuck in a rut, and won't be able to rise to a higher level of sophistication.
I was intrigued that his hierarchy might relate to the building blocks of my philosophy, which are cognitive frameworks that I call "divisions of everything". I write about them in my presentation, Time and Space as Representations of Decision-Making
. My thought was that these classifiers might be distinguished by an increasing number of perspectives, from one to seven. I visited him a few times but I didn't manage to interest him in this and so I left it at that.
In recent years, I have been looking for similar frameworks in mathematics, and thus have been studying Bott periodicity and Clifford algebras, because they manifest an eight-cycle, as do the divisions of everything. It seems plausible that these structures express what happens as we add perspective after perspective. In particular, n-spheres may be related to assemblies of perspectives.
I have a B.S. in Physics and a Ph.D. in Math. So I know some basics of statistics from my Physics courses. But I would have to study the classifiers to understand what they are and what they do. But they generally same to be quadratic functions. So I thought they might be related to Clifford algebras. I am curious if you have thoughts about Clifford algebras and/or geometric algebra.
Would any of this be of interest to you? And why? I am curious.