It looks like you're new here. If you want to get involved, click one of these buttons!

- All Categories 2.3K
- Chat 499
- Study Groups 18
- Petri Nets 9
- Epidemiology 3
- Leaf Modeling 1
- Review Sections 9
- MIT 2020: Programming with Categories 51
- MIT 2020: Lectures 20
- MIT 2020: Exercises 25
- MIT 2019: Applied Category Theory 339
- MIT 2019: Lectures 79
- MIT 2019: Exercises 149
- MIT 2019: Chat 50
- UCR ACT Seminar 4
- General 67
- Azimuth Code Project 110
- Statistical methods 3
- Drafts 2
- Math Syntax Demos 15
- Wiki - Latest Changes 3
- Strategy 113
- Azimuth Project 1.1K
- - Spam 1
- News and Information 147
- Azimuth Blog 149
- - Conventions and Policies 21
- - Questions 43
- Azimuth Wiki 708

Options

This new page is, so far, mainly an annotated list of references on the entropy maximization and entropy minimization principles in nonequilibrium statistical mechanics:

Someday I hope to understand this subject, but I don't understand it yet!

## Comments

`John, I don’t understand this stuff either. But when I see a theory that seems to imply that the climate does not depend on rotation rate, say, and I know that the climate in many kinds of models that I work with does, in fact, depend on rotation rate (not surprisingly), I tend to lose interest pretty quickly! The model is a dynamical system too. If the theory doesn’t apply to it, but does apply to the actual climate, you have to explain why. When I (and others) point this out, it does not seem to have any effect at all on the small group that takes this stuff seriously. I would love to be proved wrong – the climate probably does maximize something -- and I am only familiar with a few of the simpler attempts to apply this kind of thing to climate.`

Isaac:

That's very sensible.

Who claims that some principle of maximum entropy production implies the climate doesn't depend on the Earth's rotation rate?

I'm approaching this subject from the highly theoretical end, trying to see if there's any consensus on entropy production beyond Ilya Prigogine's famous result on how entropy production is

minimizedby certain systems close to equilibrium. So far there seems to be very little consensus past this point. So, I haven't actually gotten around to looking at supposed applications to climate science.It's not obvious to me that a carefully phrased entropy maximization principle would imply that the Earth's climate is independent of its rotation rate.

It's obvious to me that a

carelesslyphrased entropy maximization principle would say that a tall sandpile wouldinstantlyflatten itself out, to maximize the rate of entropy production. Jaynes' "Principle of Maximum Caliber" seems to avoid this kind of error - but only by virtue of being so clever that it seems very difficult to extract useful information from it!It sounds like some people are over-eager to find a powerful principle to help them with the complexities of climate, and not thinking hard enough about whether their principle is true.

`Isaac: > But when I see a theory that seems to imply that the climate does not depend on rotation rate, say, and I know that the climate in many kinds of models that I work with does, in fact, depend on rotation rate (not surprisingly), I tend to lose interest pretty quickly! That's very sensible. > When I (and others) point this out, it does not seem to have any effect at all on the small group that takes this stuff seriously. Who claims that some principle of maximum entropy production implies the climate doesn't depend on the Earth's rotation rate? I'm approaching this subject from the highly theoretical end, trying to see if there's any consensus on entropy production beyond Ilya Prigogine's famous result on how entropy production is _minimized_ by certain systems close to equilibrium. So far there seems to be very little consensus past this point. So, I haven't actually gotten around to looking at supposed applications to climate science. It's not obvious to me that a carefully phrased entropy maximization principle would imply that the Earth's climate is independent of its rotation rate. It's obvious to me that a _carelessly_ phrased entropy maximization principle would say that a tall sandpile would _instantly_ flatten itself out, to maximize the rate of entropy production. Jaynes' "Principle of Maximum Caliber" seems to avoid this kind of error - but only by virtue of being so clever that it seems very difficult to extract useful information from it! It sounds like some people are over-eager to find a powerful principle to help them with the complexities of climate, and not thinking hard enough about whether their principle is true.`

I added a very general proof of Prigogine's theorem, together with some commentary, to

Extremal principles in non-equilibrium thermodynamics

(I also changed the title of this page, to make it include principles like the principle of least action and another much more controversial principle: the 'constructal law'.)

`I added a very general proof of Prigogine's theorem, together with some commentary, to [[Extremal principles in non-equilibrium thermodynamics]] (I also changed the title of this page, to make it include principles like the principle of least action and another much more controversial principle: the 'constructal law'.)`

Australian Atmospheric Scientist Garth Paltridge got pretty good results for Earth's climate using maxEP while working for CSIRO in the late 70s. I vaguely remember that he made a good prediction for Titan's climate. I tried to find out why CSIRO didn't stick this on its list of scientific achievements.

`Australian Atmospheric Scientist [Garth Paltridge](http://en.wikipedia.org/wiki/Garth_Paltridge) got pretty good results for Earth's climate using maxEP while working for CSIRO in the late 70s. I vaguely remember that he made a good prediction for Titan's climate. I tried to find out why CSIRO didn't stick this on its list of scientific achievements.`

`Paltridge's is one example of a theory that does not depend on rotation rate.`

So, being sure John will one day understand this stuff and write more of his great expositions in the process, I have some hope that someday I'll understand something, too. :-)

Alas I haven't yet managed to print and read Kleidon's article which I quoted on the Gaia wiki page:

`So, being sure John will one day understand this stuff and write more of his great expositions in the process, I have some hope that someday I'll understand something, too. :-) Alas I haven't yet managed to print and read Kleidon's article which I quoted on the [[Gaia]] wiki page: * Kleidon, A. 2009: [Nonequilibrium thermodynamics and maximum entropy production in the Earth system. Applications and implications](http://www.springerlink.com/content/f3561w2k801u6215/fulltext.pdf), Naturwissenschaften Volume 96, Number 6, 653-677 > nonequilibrium thermodynamics and MEP show great promise in allowing us to formulate a quantifiable, holistic perspective of the Earth system at a fundamental level. This perspective would allow us to understand how the Earth system organizes itself in its functioning, how it reacts to change, and how it has evolved through time.`

I've encountered a "Kohler's variational principle" in kinetic theory. From the book "Mathematical theory of transport processes in gases",1972, by Ferziger J.H. and Kaper H.G. :

As far as I understand it was proved within Chapman–Enskog theory.

In the book

Statistical Mechanics of Nonequilibrium Processes, 1996 by D. N. Zubarev, V. G. Morozov, G. Röpke, in the chapter's 5 appendix another (more general?) principle is discussed, that is claimed to be similar to the Kohler's variational principle.Unfortunately at present I can't say anything more, so those are just kind of links.

`I've encountered a "Kohler's variational principle" in kinetic theory. From the book "Mathematical theory of transport processes in gases",1972, by Ferziger J.H. and Kaper H.G. : <blockquote><p>Thus we can formulate the following maximum principle: In non-equilibrium systems the distribution of the molecular velocities is such that, for given temperature and velocity gradients, the rate of change of the entropy density due to collisions is as large as possible. This maximum principle, together with a similar minimum principle, was first given by Kohler [1948]. A discussion of these and other variational principles can be found in a paper by Ziman [1956] or in a paper by Snider [1964a].</p></blockquote> As far as I understand it was proved within Chapman–Enskog theory. In the book _Statistical Mechanics of Nonequilibrium Processes_, 1996 by D. N. Zubarev, V. G. Morozov, G. Röpke, in the chapter's 5 appendix another (more general?) principle is discussed, that is claimed to be similar to the Kohler's variational principle. Unfortunately at present I can't say anything more, so those are just kind of links.`

I just spent an enlightening week with Eric Smith. He has a nice and recent review paper (which I should be clear I have not completed reading) that I think could take anyone with some background in physics relatively far in understanding the topics mentioned here:

74(2011), 046601.If anyone does choose to read and understand it, I'd be interested to see comments or questions that it evokes. This is, of course, not least because I would like to deepen my own understanding of this field.

`I just spent an enlightening week with [Eric Smith](http://tuvalu.santafe.edu/~desmith). He has a nice and recent review paper (which I should be clear I have not completed reading) that I think could take anyone with some background in physics relatively far in understanding the topics mentioned here: * Eric Smith, [Large-deviation principles, stochastic effective actions, path entropies, and the structure and meaning of thermodynamic descriptions](http://arxiv.org/abs/1102.3938), [Reports on Progress in Physics](http://dx.doi.org/10.1088/0034-4885/74/4/046601), **74** (2011), 046601. If anyone does choose to read and understand it, I'd be interested to see comments or questions that it evokes. This is, of course, not least because I would like to deepen my own understanding of this field.`

Thanks very much for all your comments and links, Robert, Isaac, Martin, Grigory and Cameron! I will add some of this material to Extremal principles in non-equilibrium thermodynamics.

It's amazing how much confusion surrounds the subject of 'maximum entropy production'. Wikipedia suggests why:

So, I am very happy to see that someone (maybe Kohler) claims to have proved maximum entropy production in some specific context (maybe the Chapman-Enskog theory of gases). This is the kind of thing that one can check, understand, and perhaps generalize.

The first question would be: maximum entropy production

subject to what constraints? Obviously a gas does not instantly leap to thermal equilibrium, which would be the naive interpretation of "maximizing the rate of entropy production".`Thanks very much for all your comments and links, Robert, Isaac, Martin, Grigory and Cameron! I will add some of this material to [[Extremal principles in non-equilibrium thermodynamics]]. It's amazing how much confusion surrounds the subject of 'maximum entropy production'. [Wikipedia](http://en.wikipedia.org/wiki/Extremal_principles_in_non-equilibrium_thermodynamics) suggests why: > According to Kondepudi (2008), and to Grandy (2008), there is no general rule that provides an extremum principle that governs the evolution of a far-from-equilibrium system to a steady state. According to Glansdorff and Prigogine (1971, page 16), irreversible processes usually are not governed by global extremal principles because description of their evolution requires differential equations which are not self-adjoint, but local extremal principles can be used for local solutions. Lebon Jou and Casas-Vásquez (2008) state that "In non-equilibrium ... it is generally not possible to construct thermodynamic potentials depending on the whole set of variables". Šilhavý (1997) offers the opinion that "... the extremum principles of thermodynamics ... do not have any counterpart for [non-equilibrium] steady states (despite many claims in the literature)." It follows that any general extremal principle for a non-equilibrium problem will need to refer in some detail to the constraints that are specific for the structure of the system considered in the problem. So, I am very happy to see that someone (maybe Kohler) claims to have proved maximum entropy production in some specific context (maybe the [Chapman-Enskog theory](http://en.wikipedia.org/wiki/Chapman%E2%80%93Enskog_theory) of gases). This is the kind of thing that one can check, understand, and perhaps generalize. The first question would be: maximum entropy production _subject to what constraints_? Obviously a gas does not instantly leap to thermal equilibrium, which would be the naive interpretation of "maximizing the rate of entropy production".`

Cameron wrote:

I don't have time to read this carefully now, and I don't find it an easy read, but I'll say one thing just so I don't forget:

The main example considered here is a classical particle moving stochastically in a double well potential. I may be mixed up, but it seems we're studying almost exactly the same system in our work on stochastic resonance, which may play a role in understanding the glacial cycles.

`Cameron wrote: > * Eric Smith, [Large-deviation principles, stochastic effective actions, path entropies, and the structure and meaning of thermodynamic descriptions](http://arxiv.org/abs/1102.3938), [Reports on Progress in Physics](http://dx.doi.org/10.1088/0034-4885/74/4/046601), **74** (2011), 046601. > If anyone does choose to read and understand it, I'd be interested to see comments or questions that it evokes. This is, of course, not least because I would like to deepen my own understanding of this field. I don't have time to read this carefully now, and I don't find it an easy read, but I'll say one thing just so I don't forget: The main example considered here is a classical particle moving stochastically in a double well potential. I may be mixed up, but it seems we're studying almost exactly the same system in our work on [stochastic resonance](http://www.azimuthproject.org/azimuth/show/Stochastic+resonance), which may play a role in understanding the glacial cycles.`

That looks like an interesting connection John. I had not yet looked at the stochastic resonance page. I will as I continue reading this paper.

`That looks like an interesting connection John. I had not yet looked at the stochastic resonance page. I will as I continue reading this paper.`

That Kohler's principle varies only the distribution of velocity, spatial distribution is not affected by maximization. And there are at least two constraints on the velocity distribution --- averages $\langle \boldsymbol v^2 \rangle$ and $\langle \boldsymbol v \rangle$ are given.

`<blockquote><p>The first question would be: maximum entropy production subject to what constraints? Obviously a gas does not instantly leap to thermal equilibrium, which would be the naive interpretation of "maximizing the rate of entropy production".</p></blockquote> That Kohler's principle varies only the distribution of velocity, spatial distribution is not affected by maximization. And there are at least two constraints on the velocity distribution --- averages $\langle \boldsymbol v^2 \rangle$ and $\langle \boldsymbol v \rangle$ are given.`

Okay, thanks!

`Okay, thanks!`

From a practical point of view, MEP works quite well in many circumstances, and this is obviously not pure chance... The Paltridge model is an illustration of that. Another example (used everyday in engineering !) is the Nusselt-Rayleigh relationship in thermal convection ( Nu = Ra^(1/3) ) so this kind of principle is indeed useful, is even used every day, even though there is no strong theoretical justification for it (...yet). There are dozens of such example.

I am interested to know if this can be useful for climatology, and I therefore worked a bit on Paltridge-like models (see Herbert et al., QJRMS 2011). I believe MEP is an elegant way to build very simple climate models, that have almost no tuning parameter... Which makes them interesting per se, even if the results are not perfect (May be I should say, in particular when the results are not perfect ?..).

Anyway, the Paltridge results are not an artifact... (Besides, MEP also works on other planets, see Ralph Lorenz' papers). First, it does not work perfectly well (there are many missing features in the temperature fields) and this can be certainly improved with more physics. The fact that it predicts a resonnably good meridional heat transport without rotation is indeed impressive, and shows that (in a sufficiently turbulent atmosphere..., with a rotation slow enough...), the planet rotation may not be so important for this particular feature (this has been examined is some details by Jupp and Cox, Philos. T. Roy. Soc. B, 365, 1355–1365, 2010). To please Isaac and many other meteorologists, I am convinced that Earth rotation (and atmospheric dynamics in general) is important for many other aspects of climate.

The great thing with MEP is that is takes care of "turbulent diffusion", which is one of the weakest point in climate models. I certainly don't think it replaces the other physical processes (energy conservation, water cycle, dynamics, ...). So I believe there is a lot of room in climatology for using MEP (at least trying to use it), together with other physical constraints.

`The use of MEP (maximum entropy production) principle leads to quite contrasted reactions in the scientific community. The proponents tend to present it as a magic bullet that can be applied to almost anything (from cosmology, to fluid dynamics in climate, to sociology...). The adversaries insist that this is not a law of Nature and therefore it cannot be used, and should not be used ! The truth is probably in between... From a practical point of view, MEP works quite well in many circumstances, and this is obviously not pure chance... The Paltridge model is an illustration of that. Another example (used everyday in engineering !) is the Nusselt-Rayleigh relationship in thermal convection ( Nu = Ra^(1/3) ) so this kind of principle is indeed useful, is even used every day, even though there is no strong theoretical justification for it (...yet). There are dozens of such example. I am interested to know if this can be useful for climatology, and I therefore worked a bit on Paltridge-like models (see Herbert et al., QJRMS 2011). I believe MEP is an elegant way to build very simple climate models, that have almost no tuning parameter... Which makes them interesting per se, even if the results are not perfect (May be I should say, in particular when the results are not perfect ?..). Anyway, the Paltridge results are not an artifact... (Besides, MEP also works on other planets, see Ralph Lorenz' papers). First, it does not work perfectly well (there are many missing features in the temperature fields) and this can be certainly improved with more physics. The fact that it predicts a resonnably good meridional heat transport without rotation is indeed impressive, and shows that (in a sufficiently turbulent atmosphere..., with a rotation slow enough...), the planet rotation may not be so important for this particular feature (this has been examined is some details by Jupp and Cox, Philos. T. Roy. Soc. B, 365, 1355–1365, 2010). To please Isaac and many other meteorologists, I am convinced that Earth rotation (and atmospheric dynamics in general) is important for many other aspects of climate. The great thing with MEP is that is takes care of "turbulent diffusion", which is one of the weakest point in climate models. I certainly don't think it replaces the other physical processes (energy conservation, water cycle, dynamics, ...). So I believe there is a lot of room in climatology for using MEP (at least trying to use it), together with other physical constraints.`

It seems the rotation thing is no big deal. Kleidon (loc. cit) says:

Kleidon's article has an impressive list "Demonstration of MEP states: examples", including ocean dynamics. (But what actually impressed me most is the pictures and diagrams, e.g. depicting water and carbon cycle in electric circuitry (only batteries and resistors).)

It seems (but me dunno much) MEP represents a terrific and completely different new area of

experimental mathematical physics.What would be the best book/article for dummies like me on variational principles in thermodynamics? (I've just ordered Lancos' Variational Principles of Mechanics. What's the shiniest thermodynamics gem to put next to it in my micro library?)

`It seems the rotation thing is no big deal. Kleidon (loc. cit) says: > Applications of MEP to atmospheric dynamics have been criticized by arguing that the solution of MEP does not account for the important role of rotation rate in the solution (Rodgers 1976; Goody 2007). This, however, is rather a limitation of the simple energy balance model used to demonstrate MEP, rather than a deficiency of the hypothesis per se. The demonstration of MEP in simulations with an atmospheric general circulation model (Kleidon et al. 2003, 2006) clearly shows that the effect of rotation rate can be included in the maximization. Kleidon's article has an impressive list "Demonstration of MEP states: examples", including ocean dynamics. (But what actually impressed me most is the pictures and diagrams, e.g. depicting water and carbon cycle in electric circuitry (only batteries and resistors).) It seems (but me dunno much) MEP represents a terrific and completely different new area of _experimental mathematical physics_. What would be the best book/article for dummies like me on variational principles in thermodynamics? (I've just ordered Lancos' Variational Principles of Mechanics. What's the shiniest thermodynamics gem to put next to it in my micro library?)`

By the way, it's possible that one reason some people are unpersuaded by the work of Garth Paltridge is that his book

The Climate Caper: Facts and Fallacies of Global Warmingquestions the consensus on global warming:I don't know if this related to his advocacy of the maximum entropy production principle! But, it's not the sort of thing that would make most climate scientists eager to pursue the maximum entropy production principle.

`By the way, it's possible that one reason some people are unpersuaded by the work of Garth Paltridge is that his book _[The Climate Caper: Facts and Fallacies of Global Warming](http://books.google.com.au/books?id=FXNzPgAACAAJ&dq=climate+caper&ei=DCDQSuylA5-qkASewLz1DQ)_ questions the consensus on global warming: > So you think the theory of disastrous climate change has been proved, and that scientists are united in their efforts to force the nations of the world to reduce their carbon emissions! In his book _The Climate Caper_, with a light touch and nicely readable manner, Professor Paltridge shows that the case for action against climate change is not so certain as is presented to politicians and the public. He leads us through the massive uncertainties which are inherently part of the 'climate modelling process'; he examines the even greater uncertainties associated with economic forecasts of climatic doom; and he discusses in detail the conscious and sub-conscious forces operating to ensure that scepticism within the scientific community is kept from the public eye. It seems that governments are indeed becoming captive to a scientific and technological elite - an elite which is achieving its ends by manipulating fear of climate change into the world's greatest example of a religion for the politically correct. I don't know if this related to his advocacy of the maximum entropy production principle! But, it's not the sort of thing that would make most climate scientists eager to pursue the maximum entropy production principle.`

Martin wrote:

This subject is so murky and controversial that it's very hard for me to say! I often think

everyoneis a dummy when it comes to variational principles in thermodynamics. I don't even know if any such principle is valid, except in the very limited linear regime famously studied by Prigogine. Some people say yes, others say no.I think you should start with the Wikipedia article, which may fill you with despair.

`Martin wrote: > What would be the best book/article for dummies like me on variational principles in thermodynamics? This subject is so murky and controversial that it's very hard for me to say! I often think _everyone_ is a dummy when it comes to variational principles in thermodynamics. I don't even know if any such principle is valid, except in the very limited linear regime famously studied by Prigogine. Some people say yes, others say no. I think you should start with the [Wikipedia article](http://en.wikipedia.org/wiki/Extremal_principles_in_non-equilibrium_thermodynamics), which may fill you with despair.`

As John said, the subject being quite controversial, a "good book/article" for some people may be a bad one for others...

Still a good review paper (ie. lying outside of the controversies) is:

- Martyushev and Seleznev. Maximum entropy production principle in physics, chemistry and biology. Phys Rep (2006) vol. 426 (1) pp. 1-45

Somewhat outside the subject, I would also mention a short paper by Marston in Physics (2011) (http://link.aps.org/doi/10.1103/Physics.4.20) who points out that statistical mechanics has obviously something to say about climate.

To John:

I do not believe that climate scientists see MEP as a controversial issue because Paltridge may have written controversial books... The main advocates of MEP (eg. Kleidon) are clearly on a quite different track. The difficulty with ME or MEP has probably more to do with physicists (and climate scientists) having difficulties to deal with "chance" or with the "unknown". Their strategy is generally to refine equations (processes, time steps,...). From the statisticians' viewpoint (eg Jaynes), it is more practical and generally more useful to deal with the absence of knowledge in a quantitative way. Most scientists are simply not used to this, statistical mechanics was not acclaimed at the time of Boltzmann, and is still now taught only in quite specific cursus (usually not in "climate" cursus). Maximum entropy is not really a "physical principle" but a way to combine "absence of knowledge" (entropy) and knowledge (constraints) to get the best out of the available information. There is no a priori garantee of success (available information may not be sufficient), but only trial and error...

`To Martin: As John said, the subject being quite controversial, a "good book/article" for some people may be a bad one for others... Still a good review paper (ie. lying outside of the controversies) is: - Martyushev and Seleznev. Maximum entropy production principle in physics, chemistry and biology. Phys Rep (2006) vol. 426 (1) pp. 1-45 Somewhat outside the subject, I would also mention a short paper by Marston in Physics (2011) (http://link.aps.org/doi/10.1103/Physics.4.20) who points out that statistical mechanics has obviously something to say about climate. To John: I do not believe that climate scientists see MEP as a controversial issue because Paltridge may have written controversial books... The main advocates of MEP (eg. Kleidon) are clearly on a quite different track. The difficulty with ME or MEP has probably more to do with physicists (and climate scientists) having difficulties to deal with "chance" or with the "unknown". Their strategy is generally to refine equations (processes, time steps,...). From the statisticians' viewpoint (eg Jaynes), it is more practical and generally more useful to deal with the absence of knowledge in a quantitative way. Most scientists are simply not used to this, statistical mechanics was not acclaimed at the time of Boltzmann, and is still now taught only in quite specific cursus (usually not in "climate" cursus). Maximum entropy is not really a "physical principle" but a way to combine "absence of knowledge" (entropy) and knowledge (constraints) to get the best out of the available information. There is no a priori garantee of success (available information may not be sufficient), but only trial and error...`

I find the comments by both Didier and John a bit strange. On the one hand, it is argued that "climate scientists" don't pay attention to MEP theory because they haven't had the appropriate courses. On the other hand, it is because of someone's views about climate change. (I hope this was a joke.) Speaking for myself, I haven't paid much attention because the versions that I have looked at don't make any sense to me -- unlike, say, the beautiful Robert-Sommeria (1991) statistical theories for the inviscid evolution of two-dimensional turbulence, or variants thereof, where the constraints are the the area of fluid with any given value of potential vorticity -- ie Venaille and Bouchet and references therein. The climate related papers that I have read typically talk about microscopic entropy (the familiar entropy of the atmosphere as an ideal gas) which has absolutely nothing to do with theories involving a macroscopic entropy like Robert-Sommeria. Yet it seems whenever anyone mentions "entropy" in a paper it is used as support for this "theory".

`I find the comments by both Didier and John a bit strange. On the one hand, it is argued that "climate scientists" don't pay attention to MEP theory because they haven't had the appropriate courses. On the other hand, it is because of someone's views about climate change. (I hope this was a joke.) Speaking for myself, I haven't paid much attention because the versions that I have looked at don't make any sense to me -- unlike, say, the beautiful Robert-Sommeria (1991) statistical theories for the inviscid evolution of two-dimensional turbulence, or variants thereof, where the constraints are the the area of fluid with any given value of potential vorticity -- ie [Venaille and Bouchet](http://arxiv.org/abs/1011.2556) and references therein. The climate related papers that I have read typically talk about microscopic entropy (the familiar entropy of the atmosphere as an ideal gas) which has absolutely nothing to do with theories involving a macroscopic entropy like Robert-Sommeria. Yet it seems whenever anyone mentions "entropy" in a paper it is used as support for this "theory".`

I was not specifically pointing at the training of "climate scientists" but more generally at "physicists"... And this is probably not a "training" issue, but something deeper and more philosophical about the use of statistics in physics. You need to make a simple assumption on what you don't know very well (equipartition in phase space for instance) and often it means neglecting some important pieces of knowledge about the micro-physics, something physicists (including myself) are not very happy to do. Of course, this is only my personal feeling....

But of course you are perfectly right about "entropy". The fundamental issue of MEP is to relate microscopic (thermodynamical) entropy production with some "information entropy" that can be maximized in the usual way. The link is the "dynamical entropy" which relates to the distribution of trajectories (in contrat to "states"). A nice starting point is the Kolmogorov-Sinai "entropy" in dynamical systems theory (which relates to the largest Lyapunov exponent) which is in fact an "entropy production" in terms of phase space... Building a rigourous framework (which can tell us when this will work and when this won't) is clearly an open question. But the situation is not hopeless, at least in simple (trivial?) cases...(Cécile Monthus J. Stat. Mech. (2011) P03008, Non-equilibrium steady states: maximization of the Shannon entropy associated with the distribution of dynamical trajectories in the presence of constraints).

`To Isaac: I was not specifically pointing at the training of "climate scientists" but more generally at "physicists"... And this is probably not a "training" issue, but something deeper and more philosophical about the use of statistics in physics. You need to make a simple assumption on what you don't know very well (equipartition in phase space for instance) and often it means neglecting some important pieces of knowledge about the micro-physics, something physicists (including myself) are not very happy to do. Of course, this is only my personal feeling.... But of course you are perfectly right about "entropy". The fundamental issue of MEP is to relate microscopic (thermodynamical) entropy production with some "information entropy" that can be maximized in the usual way. The link is the "dynamical entropy" which relates to the distribution of trajectories (in contrat to "states"). A nice starting point is the Kolmogorov-Sinai "entropy" in dynamical systems theory (which relates to the largest Lyapunov exponent) which is in fact an "entropy production" in terms of phase space... Building a rigourous framework (which can tell us when this will work and when this won't) is clearly an open question. But the situation is not hopeless, at least in simple (trivial?) cases...(Cécile Monthus J. Stat. Mech. (2011) P03008, Non-equilibrium steady states: maximization of the Shannon entropy associated with the distribution of dynamical trajectories in the presence of constraints).`

It's not a joke.

Iinstantly became more suspicious of Paltridge's work, not knowing any of the details, when I discovered that he wrote a book claiming that "governments are indeed becoming captive to a scientific and technological elite - an elite which is achieving its ends by manipulating fear of climate change into the world's greatest example of a religion for the politically correct". I think this is a perfectly rational reaction on my part. Of course his political views don't prove his ideas on climate physics are wrong. But I believe there's a demonstrable correlation between such views and strange theories of climate physics, so as a matter of probabilistic reasoning I think oneshouldbecome more suspicious of someone's physics when one discovers they've written a book like this.Of course you, knowing more detailed information, can rely less on such rough heuristics.

`> On the other hand, it is because of someone's views about climate change. (I hope this was a joke.) It's not a joke. _I_ instantly became more suspicious of Paltridge's work, not knowing any of the details, when I discovered that he wrote a book claiming that "governments are indeed becoming captive to a scientific and technological elite - an elite which is achieving its ends by manipulating fear of climate change into the world's greatest example of a religion for the politically correct". I think this is a perfectly rational reaction on my part. Of course his political views don't prove his ideas on climate physics are wrong. But I believe there's a demonstrable correlation between such views and strange theories of climate physics, so as a matter of probabilistic reasoning I think one _should_ become more suspicious of someone's physics when one discovers they've written a book like this. Of course you, knowing more detailed information, can rely less on such rough heuristics.`

First, Paltridge is not at the origin of the MEP ideas: this goes back to many previous attempts (Ziman, Ziegler and others) to apply statistical mechanics to non-equilibrium systems. Paltridge was looking for an extremal principle that could explain climate and found (very empirically) that something which looked like entropy production could do the job (actually he did not recognised it at first for what it was: entropy production). There is NO theory behind his papers, just an empirical result (...maximize this, you get that. I did the computing: it is very easy and it works). The "strange theories" using MEP have been built by others, and not only in the climate area (actually, Kleidon and Dewar are much more ecosystem scientists than climate scientists for instance, Ralph Lorenz is an astrophysicist, and so on ...). Paltridge has never done theoretical work on that topic, but just provided a very nice example that, indeed, it may work (at least in some circumstances). He has not (to my knowledge) published any new result on this topic since the end of the 1970s, a long time before climate change became a political issue. I therefore don't see any possible connection between MEP and his personal views on global change...though actually I don't know him and I never met him.

`John, you should put things a bit more in context... First, Paltridge is not at the origin of the MEP ideas: this goes back to many previous attempts (Ziman, Ziegler and others) to apply statistical mechanics to non-equilibrium systems. Paltridge was looking for an extremal principle that could explain climate and found (very empirically) that something which looked like entropy production could do the job (actually he did not recognised it at first for what it was: entropy production). There is NO theory behind his papers, just an empirical result (...maximize this, you get that. I did the computing: it is very easy and it works). The "strange theories" using MEP have been built by others, and not only in the climate area (actually, Kleidon and Dewar are much more ecosystem scientists than climate scientists for instance, Ralph Lorenz is an astrophysicist, and so on ...). Paltridge has never done theoretical work on that topic, but just provided a very nice example that, indeed, it may work (at least in some circumstances). He has not (to my knowledge) published any new result on this topic since the end of the 1970s, a long time before climate change became a political issue. I therefore don't see any possible connection between MEP and his personal views on global change...though actually I don't know him and I never met him.`

Does anybody know a really simple example where MEP is valid? So one can fully resolve the system behavior without employing the MEP and then compare with the MEP solution. On the other hand MEP seems to be formulated/valid for really complex systems, it is its key characteristic.

I'm still struggling to get MEP's formulation, an example would really help, and there is no such thing as "too simple example".

`Does anybody know a really simple example where MEP is valid? So one can fully resolve the system behavior without employing the MEP and then compare with the MEP solution. On the other hand MEP seems to be formulated/valid for really complex systems, it is its key characteristic. I'm still struggling to get MEP's formulation, an example would really help, and there is no such thing as "too simple example".`

I am somewhat sympathetic to Paltridge's views quoted here. The papers I've read and the data I've seen leave a lot of room for skepticism. I do get a feeling people tend to hyperventilate and view any skepticism in an extremely negative light.

I don't doubt the climate is changing and I don't doubt the affects can be devasting. I don't even doubt that human behavior is accelerating the process. However, I tend to think the climate change would happen anyway. Rather than attempting to slow the inevitable, we should accept changes are coming and prepare for it.

My opinion is still evolving, but that is what I currently think.

`I am somewhat sympathetic to Paltridge's views quoted here. The papers I've read and the data I've seen leave a lot of room for skepticism. I do get a feeling people tend to hyperventilate and view any skepticism in an extremely negative light. I don't doubt the climate is changing and I don't doubt the affects can be devasting. I don't even doubt that human behavior is accelerating the process. However, I tend to think the climate change would happen anyway. Rather than attempting to slow the inevitable, we should accept changes are coming and prepare for it. My opinion is still evolving, but that is what I currently think.`

This thread is on the article about extremal principles for nonequilibrium thermodynamics, so I won't reply to #25 here, even though it makes me want to say stuff.

`This thread is on the article about extremal principles for nonequilibrium thermodynamics, so I won't reply to #25 here, even though it makes me want to say stuff.`

A simple and illustrative example of "MEP validity" is the Rayleigh-Benard convection (a fluid between a hot and a cold plate). In this particular case, a very simple (and practical) law to relate the Nusselt number Nu (the heat transport in dimensionless form) and the Rayleigh number Ra (the temperature gradient) is a power law obtained by Malkus (in 1954) by maximizing the heat transport (this is here equivalent to entropy production since boundary temperatures are fixed), with the constraint that there should be a (marginally stable) boundary layer (diffusive layer) at the top and the bottom of the fluid. Then you get very easily Nu proportional to Ra^(1/3) which is a reasonably good approximation over many orders of magnitude for Ra (actually, over the whole experimental range, except may be at low Ra... see below).

Malkus scaling is only an approximation, but this is precisely the spirit of MEP: how to get a simple (approximate) answer with minimal assumptions. MEP is not going to replace the many important physical details of fluid mechanics. On the contrary, it is a way to find a simple, approximate, and possibly robust answer from the most critical constraints on the problem. There is a large number of experiments and theories on the Rayleigh-Benard convection, and to my knowledge, no theoretical justification to Malkus' theory, beyond empirical observation, or beyond a statement like "maximize transport and you get approximately the right answer". The deviations between experiment (or more sophisticated theories, or numerical simulations) and the Malkus scaling are larger when the convection is "gentle" (small Ra), and smaller in the highly turbulent regimes (high Ra). This is consistent with the idea of MEP that turbulence is mixing things as efficiently as possible (so it works better for highly turbulent regimes).

More philosophically: I do not think that MEP should be "valid" or "invalid". I prefer to say that it is a procedure that may be useful in some cases ... The physics is not in MEP, but in the constraints (see my comment above #19).

`To Grigory: A simple and illustrative example of "MEP validity" is the Rayleigh-Benard convection (a fluid between a hot and a cold plate). In this particular case, a very simple (and practical) law to relate the Nusselt number Nu (the heat transport in dimensionless form) and the Rayleigh number Ra (the temperature gradient) is a power law obtained by Malkus (in 1954) by maximizing the heat transport (this is here equivalent to entropy production since boundary temperatures are fixed), with the constraint that there should be a (marginally stable) boundary layer (diffusive layer) at the top and the bottom of the fluid. Then you get very easily Nu proportional to Ra^(1/3) which is a reasonably good approximation over many orders of magnitude for Ra (actually, over the whole experimental range, except may be at low Ra... see below). Malkus scaling is only an approximation, but this is precisely the spirit of MEP: how to get a simple (approximate) answer with minimal assumptions. MEP is not going to replace the many important physical details of fluid mechanics. On the contrary, it is a way to find a simple, approximate, and possibly robust answer from the most critical constraints on the problem. There is a large number of experiments and theories on the Rayleigh-Benard convection, and to my knowledge, no theoretical justification to Malkus' theory, beyond empirical observation, or beyond a statement like "maximize transport and you get approximately the right answer". The deviations between experiment (or more sophisticated theories, or numerical simulations) and the Malkus scaling are larger when the convection is "gentle" (small Ra), and smaller in the highly turbulent regimes (high Ra). This is consistent with the idea of MEP that turbulence is mixing things as efficiently as possible (so it works better for highly turbulent regimes). More philosophically: I do not think that MEP should be "valid" or "invalid". I prefer to say that it is a procedure that may be useful in some cases ... The physics is not in MEP, but in the constraints (see my comment above #19).`

Great! Looks like Malkus' theory is the paradigmatic example of the sort of experimental mathematical physics I was dreaming of.

Alas I've never got into numerics (except 2 boring Fortran classes and writing the world's first ray tracing graphic in BASIC for toy computers at end of high school). But I have dreams. One is recycling rounding errors in floating point random variables and doing stochastic calculus numerics...

`Great! Looks like Malkus' theory is the paradigmatic example of the sort of experimental mathematical physics I was dreaming of. --------------------- > The difficulty with ME or MEP has probably more to do with physicists (and climate scientists) having difficulties to deal with "chance" or with the "unknown". Their strategy is generally to refine equations (processes, time steps,...). From the statisticians' viewpoint (eg Jaynes), it is more practical and generally more useful to deal with the absence of knowledge in a quantitative way. Alas I've never got into numerics (except 2 boring Fortran classes and writing the world's first ray tracing graphic in BASIC for toy computers at end of high school). But I have dreams. One is recycling rounding errors in floating point random variables and doing stochastic calculus numerics...`

Didier wrote:

And what is the name of that Malkus's work? Is it "Discrete transitions in turbulent convection" Proc. R. Soc. Lond. A 1954 225, 185-195? Anyway where I can read about the Malkus's analysis itself?

UPDThe article in question is actually dated to 1958, "Finite amplitude cellular convection" by Malkus and Veronis, DOI: 10.1017/S0022112058000410`Didier wrote: >In this particular case, a very simple (and practical) law to relate the Nusselt number Nu (the heat transport in dimensionless form) and the Rayleigh number Ra (the temperature gradient) is a power law obtained by Malkus (in 1954) by maximizing the heat transport (this is here equivalent to entropy production since boundary temperatures are fixed), with the constraint that there should be a (marginally stable) boundary layer (diffusive layer) at the top and the bottom of the fluid. And what is the name of that Malkus's work? Is it "Discrete transitions in turbulent convection" Proc. R. Soc. Lond. A 1954 225, 185-195? Anyway where I can read about the Malkus's analysis itself? **UPD** The article in question is actually dated to 1958, "Finite amplitude cellular convection" by Malkus and Veronis, DOI: 10.1017/S0022112058000410`

I am sorry, but I also don't have access to the original papers of Malkus... The 1/3 power law is usually refered to the following paper:

Malkus, W. V. R., The heat transport and spectrum of thermal turbulence, Proc. R. Soc. London A, 225, 196 –212, 1954

The analysis is quite simple. The problem is basically 1-D (at least for mean values) and the heat flux through the fluid has to be the same throughout the fluid, and in particular through the boundaries. The maximum flux is therefore the one for which the temperature is constant in the bulk, and linear in the boundary layers (because of the law of diffusion) but marginally stable. There, the size d of the boundary layer is related to the temperature gradient and to the critical Rayleigh number Ra* by:

d ~ (Ra/∆T)^(1/3) ~ (Ra/Ra)^(1/3) (from the definitions of Ra* and Ra).

and the Nusselt number

Nu ~ HeatFlux/(k ∆T) = (k ∆T/2d)/(k ∆T) ~ (Ra/Ra*)^(1/3)

There is nothing more than that.

Of course it is only an approximation or more precisely an upper limit, since other physical constraints could restrict furthermore the heat flux or the fluid flow.

There has been some debate recently on these kinds of scalings at very high Rayleigh numbers with larger exponents (eg. there is a theory for exponent = 1/2). But in Niemela et al. (Turbulent convection at very high Rayleigh numbers. Nature (2000) vol. 404 pp. 837-840), their best fit is

Nu = 0.124 Ra^0.309 over 11 orders of magnitudes.

This is amazingly close (at least it seems to me...) to the 1/3 exponent given by the simplicity of the argument above.

`Grigory: I am sorry, but I also don't have access to the original papers of Malkus... The 1/3 power law is usually refered to the following paper: Malkus, W. V. R., The heat transport and spectrum of thermal turbulence, Proc. R. Soc. London A, 225, 196 –212, 1954 The analysis is quite simple. The problem is basically 1-D (at least for mean values) and the heat flux through the fluid has to be the same throughout the fluid, and in particular through the boundaries. The maximum flux is therefore the one for which the temperature is constant in the bulk, and linear in the boundary layers (because of the law of diffusion) but marginally stable. There, the size d of the boundary layer is related to the temperature gradient and to the critical Rayleigh number Ra* by: d ~ (Ra*/∆T)^(1/3) ~ (Ra*/Ra)^(1/3) (from the definitions of Ra* and Ra). and the Nusselt number Nu ~ HeatFlux/(k ∆T) = (k ∆T/2d)/(k ∆T) ~ (Ra/Ra*)^(1/3) There is nothing more than that. Of course it is only an approximation or more precisely an upper limit, since other physical constraints could restrict furthermore the heat flux or the fluid flow. There has been some debate recently on these kinds of scalings at very high Rayleigh numbers with larger exponents (eg. there is a theory for exponent = 1/2). But in Niemela et al. (Turbulent convection at very high Rayleigh numbers. Nature (2000) vol. 404 pp. 837-840), their best fit is Nu = 0.124 Ra^0.309 over 11 orders of magnitudes. This is amazingly close (at least it seems to me...) to the 1/3 exponent given by the simplicity of the argument above.`

I added a brief discussion of

Clim. Dyn.38(2012), 1211–1227.I couldn't find this page at first, so I added a redirect from Maximum entropy production. To do this sort of thing, you add a line like

`[[!redirects Maximum entropy production]]`

at the end.

`I added a brief discussion of * Salvatore Pascale, Jonathan M. Gregory, Maarten H. P. Ambaum and Rémi Tailleux, [A parametric sensitivity study of entropy production and kinetic energy dissipation using the FAMOUS AOGCM](http://link.springer.com/content/pdf/10.1007%2Fs00382-011-0996-2), _Clim. Dyn._ **38** (2012), 1211–1227. I couldn't find this page at first, so I added a redirect from [[Maximum entropy production]]. To do this sort of thing, you add a line like `[[!redirects Maximum entropy production]]` at the end.`

I added a section

to Extremal principles in non-equilibrium thermodynamics. It has lots of references, but not yet links.

I like very much that he says:

On a cautionary note, Tomate (whom I tend to trust, despite this silly name) wrote:

This paper is probably

`I added a section * [Roderick Dewar on maximum entropy production](http://www.azimuthproject.org/azimuth/show/Extremal+principles+in+non-equilibrium+thermodynamics#Dewar) to [[Extremal principles in non-equilibrium thermodynamics]]. It has lots of references, but not yet links. I like very much that he says: > (...) in the absence of a fundamental explanation for MEP, it has remained something of a scientific curiosity. > Our aim is to elucidate the theoretical basis of MEP in order to underpin and guide its wider practical application. We are exploring the idea that MEP can be derived from the fundamental rules of statistical mechanics developed in physics by Boltzmann, Gibbs and Jaynes – implying that MEP is a statistical principle that describes the most likely properties of non-equilibrium systems. On a cautionary note, Tomate (whom I tend to trust, despite this silly name) wrote: > I went through Dewar’s paper some time ago. While I think most of his arguments are correct, still I don’t regard them as a full proof of the principle he has in mind. Unfortunately, he doesn’t explain analogies, differences and misunderstandings around minimum entropy production and maximum entropy production. In fact, nowhere in his articles does a clear-cut definition of MEP appear. This paper is probably * R. C. Dewar, [Maximum entropy production and the fluctuation theorem](http://www.swarmagents.cn/thesis/doc/jake_210.pdf).`

I. W. Richardson has an interesting letter on minimum entropy production versus steady state, which could probably be used to formulate these ideas using differential forms and Laplacians, or more general elliptic operators. He mentions a no-go theorem due to Gage, saying that steady state cannot always be described using an extremal principle:

Unfortunately I find this paper hard to understand, perhaps because it uses supposedly standard notation without explaining it. I've added this link to Extremal principles in nonequilibrium thermodynamics.

`I. W. Richardson has an interesting letter on minimum entropy production versus steady state, which could probably be used to formulate these ideas using differential forms and Laplacians, or more general elliptic operators. He mentions a no-go theorem due to Gage, saying that steady state cannot always be described using an extremal principle: * I. W. Richardson, [On the principle of minimum entropy production](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1367432/pdf/biophysj00690-0151.pdf), _[Biophysical Journal](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1367432/) *9* (1969), 265–267. Unfortunately I find this paper hard to understand, perhaps because it uses supposedly standard notation without explaining it. I've added this link to [[Extremal principles in nonequilibrium thermodynamics]].`

I updated the references to Roderick Dewar's work on the principle of maximum entropy production.

`I updated the references to Roderick Dewar's work on the principle of maximum entropy production.`