It looks like you're new here. If you want to get involved, click one of these buttons!

- All Categories 2.3K
- Chat 499
- Study Groups 18
- Petri Nets 9
- Epidemiology 3
- Leaf Modeling 1
- Review Sections 9
- MIT 2020: Programming with Categories 51
- MIT 2020: Lectures 20
- MIT 2020: Exercises 25
- MIT 2019: Applied Category Theory 339
- MIT 2019: Lectures 79
- MIT 2019: Exercises 149
- MIT 2019: Chat 50
- UCR ACT Seminar 4
- General 67
- Azimuth Code Project 110
- Statistical methods 3
- Drafts 2
- Math Syntax Demos 15
- Wiki - Latest Changes 3
- Strategy 113
- Azimuth Project 1.1K
- - Spam 1
- News and Information 147
- Azimuth Blog 149
- - Conventions and Policies 21
- - Questions 43
- Azimuth Wiki 708

Options

## Comments

Interestingly, a statistician colleague recently forwarded me the Silver et al. manuscript on quantum entropy, along with some related tech reports by the same authors. He was interested in possibly resurrecting their unpublished work; a top statistics journal was interested in an early version years ago, but the authors never revised it.

I only skimmed it and didn't follow the details, but I gather that the mathematical properties of quantum entropy act as some kind of "regularization" (e.g. imposing smoothness constraints) on maximum entropy inference. I told him that to write something up, he'd probably have to compare it against modern methods (e.g., in image processing spatial Bayesian models estimated with Gibbs sampling have emerged in subsequent years as competitors to MaxEnt). I'm not sure how in-vogue MaxEnt still is nowadays. I also mentioned I knew a blog where people discuss quantum entropy ...

`Interestingly, a statistician colleague recently forwarded me the Silver et al. manuscript on quantum entropy, along with some related tech reports by the same authors. He was interested in possibly resurrecting their unpublished work; a top statistics journal was interested in an early version years ago, but the authors never revised it. I only skimmed it and didn't follow the details, but I gather that the mathematical properties of quantum entropy act as some kind of "regularization" (e.g. imposing smoothness constraints) on maximum entropy inference. I told him that to write something up, he'd probably have to compare it against modern methods (e.g., in image processing spatial Bayesian models estimated with Gibbs sampling have emerged in subsequent years as competitors to MaxEnt). I'm not sure how in-vogue MaxEnt still is nowadays. I also mentioned I knew a blog where people discuss quantum entropy ...`

I only skimmed it too. I came across the Silver reference in Bayesian Methods by Leonard and Hsu, pp249-250. It says Silverman (not Silver) and Leonard have suggested regularizing a density $g$ by minimising things including terms like

$$ \int \left[ \frac{\partial^2 g}{\partial x^2} \right]^2 d x $$ which is quantum-theory-ish.

`I only skimmed it too. I came across the Silver reference in Bayesian Methods by Leonard and Hsu, pp249-250. It says Silverman (not Silver) and Leonard have suggested regularizing a density $g$ by minimising things including terms like $$ \int \left[ \frac{\partial^2 g}{\partial x^2} \right]^2 d x $$ which is quantum-theory-ish.`

I guess one question is how much does quantum math bring to the problem? Does it suggest regularization terms that are different/better than what already exists? Smoothness penalty terms aren't new (nowadays, at least).

`I guess one question is how much does quantum math bring to the problem? Does it suggest regularization terms that are different/better than what already exists? Smoothness penalty terms aren't new (nowadays, at least).`

The blog post says:

I am not sure I see the problem. A die has to be extremely biased to have m=1.01, and that is what the MEP produces. What this seems to indicate is that one would want to make doubly sure that the true mean really is 1.01 before using it to estimate the distribution. Hardcore Bayesians would probably say that the prior for m=1.01 is extremely low.

The method proposed in the article estimates the distribution as

(.9954333,.002,.001,.0006667,.0005,.0004)

while the MEP estimate is

{0.990, 0.00980, 0.0000971, 9.61

10^-7, 9.5110^-9, 9.42*10^-11}The alternative estimate balances the probabilities of the 2-5 rolls more, but at the expense of halving the probability of getting any of them, since the probability of 1 goes from .990 to .995. This means that the average "surprise" per roll is lower with the proposed alternative since the risk when betting on 1 is halved (and of course because average surprise is what MEP maximizes). So the proposed alternative really does assume more, at least according to this criterion.

`The blog post says: > A classic example is that you have some sort of device or process which can produce the numbers 1 to 6 with probabilities p1…p6, and the only thing you know about the device or process is that the mean is m. You can then use the principle of maximum entropy to choose the pi. The result is of form pi=αβi and using ∑pi=1 and ∑ipi=m you can solve (numerically) for α and β. > Why dangerous? Suppose m=1.01. The result has p6≈1e-10. I am not sure I see the problem. A die has to be extremely biased to have m=1.01, and that is what the MEP produces. What this seems to indicate is that one would want to make doubly sure that the true mean really is 1.01 before using it to estimate the distribution. Hardcore Bayesians would probably say that the prior for m=1.01 is extremely low. The method proposed in the article estimates the distribution as (.9954333,.002,.001,.0006667,.0005,.0004) while the MEP estimate is {0.990, 0.00980, 0.0000971, 9.61*10^-7, 9.51*10^-9, 9.42*10^-11} The alternative estimate balances the probabilities of the 2-5 rolls more, but at the expense of halving the probability of getting any of them, since the probability of 1 goes from .990 to .995. This means that the average "surprise" per roll is lower with the proposed alternative since the risk when betting on 1 is halved (and of course because average surprise is what MEP maximizes). So the proposed alternative really does assume more, at least according to this criterion.`

By the way, it's not a blog post - it's one of the many Experiments pages on the Azimuth Wiki, where individuals pursue little research projects. I'm eager for people to refine these into Blog articles in progress so we can post them.

`By the way, it's not a blog post - it's one of the many [[Experiments]] pages on the Azimuth Wiki, where individuals pursue little research projects. I'm eager for people to refine these into [[Blog articles in progress]] so we can post them.`