How can information (Shannon) entropy decrease ?

[Genady]

On 3/22/2022 at 2:31 AM, joigus said:

... for the 

P(α) to be well-defined probabilities they should be some fixed frequency ratios to be checked in an infinitely-many-times reproducible experiment. So this −∫P(α)lnP(α) already has to be the sought-for entropy, and not some kind of weird thing −∫P(α,t)lnP(α,t) .

IOW, how can you even have anything like time-dependent probabilities?

The book that has been mentioned by @studiot earlier (Lemons, Don S.. A Student's Guide to Entropy) has a paragraph about time-dependent entropy:

Quote

... a broader interpretation of the relative entropy according to which the multiplicity Ω is that of an instantaneous macrostate as a system evolves from an initial low entropy reference macrostate toward an equilibrium macrostate. This interpretation assumes that the system always occupies a macrostate at every instant of its evolution. In this case, S(Ω) where (2.18)  is the instantaneous entropy of a system, not necessarily in equilibrium, relative to the entropy S(Ωo) of the system in a reference macrostate. Generally, the evolution of an isolated system is in the direction of increasing entropy. ... the instantaneous macrostate multiplicity Ω and hence the instantaneous entropy S(Ω) fluctuate as the system occupies different, not quite optimal, macrostates.

 

Edited March 25, 2022 by Genady

[joigus]

On 3/22/2022 at 5:17 PM, Genady said:

I don't know how deep it is, but I doubt that I'm overthinking this. I just follow the Weinberg's derivation of the H-theorem, which in turn follows the Gibbs' one, which in turn is a generalization of the Boltzmann's one.

The first part of this theorem is that dH/dt ≤ 0. [...]

Actually, I don't think you're overthinking this at all. It does go deep. The \( P\left(\alpha,t\right) \)'s must be a priori probabilities or propensities and do not correspond to equilibrium states at all.

On 3/25/2022 at 6:39 PM, Genady said:

The book that has been mentioned by @studiot earlier (Lemons, Don S.. A Student's Guide to Entropy) has a paragraph about time-dependent entropy:

Quote

I'm not familiar with this way of dealing with the problem. The way I'm familiar with is this:

These P(α,t) 's can be hypothesized by applying the evolution equations (classical or quantum) to provide you with quantities that play the role of evolving probabilities. This is developed in great detail, e.g., in Balescu. You attribute time-independent probabilities to the (e.g. classical mechanics) initial conditions, and then you feed those initial conditions into the equations of motion to produce a time flow:

\[q_{0},p_{0}\overset{\textrm{time map}}{\mapsto}q_{t},p_{t}=q\left(q_{0},p_{0};t\right),p\left(q_{0},p_{0};t\right)\]

That induces time map onto dynamical functions:

\[A\left(q_{0},p_{0}\right)\overset{\textrm{time map}}{\mapsto}A\left(q_{t},p_{t}\right) \]

So dynamics changes dynamical functions, not probabities.

But here's the clever bit: We now do a tradeoff between averages of time-evolving variables weighed against a fixed probability density of initial conditions, and averages of fixed variables weighed against a time-evolving probability density on phase space. All of this is done by introducing a so-called Liouville operator that can be written in terms of the time-translation generator, the Hamiltonian, through the Poisson bracket. This Liouville operator produces the evolution of dynamical functions:

\[\left\langle A\right\rangle \left(t\right)=\int dq_{0}dp_{0}\rho\left(q_{0},p_{0}\right)e^{-L_{0}t}A\left(q_{0},p_{0}\right)\]

Because of the properties of this Liouville operator, you can swap its action by using integration by parts on this first-order Liouville differential operator and get,

\[\int dq_{0}dp_{0}e^{L_{0}t}\rho\left(q_{0},p_{0}\right)A\left(q_{0},p_{0}\right)=\int dq_{0}dp_{0}\rho\left(q_{0},p_{0}\right)e^{-L_{0}t}A\left(q_{0},p_{0}\right)\]

You can think of \(e^{L_{0}t}\rho\left(q_{0},p_{0}\right)\) a time-dependent probability density; while \(e^{-L_{0}t}A\left(q_{0},p_{0}\right)\) can be thought of as time-dependent dynamical functions.

For all of this to work, the system should be well-behaved; meaning that it must be ergodic. Ergodic systems are insensitive to initial conditions.

I'm sure my post leaves a lot to be desired, but I've been kinda busy lately.

 

 

 

 

 

Edited March 28, 2022 by joigus
Latex editing

[Genady]

3 hours ago, joigus said:

Actually, I don't think you're overthinking this at all. It does go deep. The P(α,t) 's must be a priori probabilities or propensities and do not correspond to equilibrium states at all.

I'm not familiar with this way of dealing with the problem. The way I'm familiar with is this:

These P(α,t) 's can be hypothesized by applying the evolution equations (classical or quantum) to provide you with quantities that play the role of evolving probabilities. This is developed in great detail, e.g., in Balescu. You attribute time-independent probabilities to the (e.g. classical mechanics) initial conditions, and then you feed those initial conditions into the equations of motion to produce a time flow:

 

q0,p0↦time mapqt,pt=q(q0,p0;t),p(q0,p0;t)

 

That induces time map onto dynamical functions:

 

A(q0,p0)↦time mapA(qt,pt)

 

So dynamics changes dynamical functions, not probabities.

But here's the clever bit: We now do a tradeoff between averages of time-evolving variables weighed against a fixed probability density of initial conditions, and averages of fixed variables weighed against a time-evolving probability density on phase space. All of this is done by introducing a so-called Liouville operator that can be written in terms of the time-translation generator, the Hamiltonian, through the Poisson bracket. This Liouville operator produces the evolution of dynamical functions:

 

〈A〉(t)=∫dq0dp0ρ(q0,p0)e−L0tA(q0,p0)

 

Because of the properties of this Liouville operator, you can swap its action by using integration by parts on this first-order Liouville differential operator and get,

 

∫dq0dp0eL0tρ(q0,p0)A(q0,p0)=∫dq0dp0ρ(q0,p0)e−L0tA(q0,p0)

 

You can think of eL0tρ(q0,p0)  a time-dependent probability density; while e−L0tA(q0,p0) can be thought of as time-dependent dynamical functions.

For all of this to work, the system should be well-behaved; meaning that it must be ergodic. Ergodic systems are insensitive to initial conditions.

I'm sure my post leaves a lot to be desired, but I've been kinda busy lately.

 

 

 

 

 

Yes, it goes deep ... and into the parts of mechanics with which I'm mostly unfamiliar. The main point is, H does not correspond to an equilibrium state until it reaches the minimum, and thus its decreasing does not correspond to the thermodynamic entropy increase.

However, the initial value of H corresponds to an equilibrium state before constraints are removed. The final value of H corresponds to a new equilibrium state, after the decrease of H. This difference between the initial and the final values of H corresponds to a difference in thermodynamic entropy between the two equilibrium states. Thus the inconsistency is removed. Banzai!

[joigus]

23 hours ago, Genady said:

Yes, it goes deep ... and into the parts of mechanics with which I'm mostly unfamiliar. The main point is, H does not correspond to an equilibrium state until it reaches the minimum, and thus its decreasing does not correspond to the thermodynamic entropy increase.

However, the initial value of H corresponds to an equilibrium state before constraints are removed. The final value of H corresponds to a new equilibrium state, after the decrease of H. This difference between the initial and the final values of H corresponds to a difference in thermodynamic entropy between the two equilibrium states. Thus the inconsistency is removed. Banzai!

Yeah, that's a beautiful idea, and I think it works. You can sub-divide also reversible processes into infinitesimally small irreversible sub-processes, and prove it for them too. I think it was very soon that people realised that entropy must be a state function, and the arguments were very similar.

I tend to look at these things more from the point of view of statistical mechanics. Boltzmann's H theorem has suffered waves of criticism through the years, but it seems to me to be quite robust.

https://en.wikipedia.org/wiki/H-theorem#Criticism_and_exceptions

I wouldn't call these criticisms "moot points," but I think all of them rest on some kind of oversimplification of physical systems.

Note that some members may find interesting: I've been watching a lecture by John Baez dealing with Shannon entropy, and the second principle of thermodynamics in the context of studying biodiversity.

It seems that people merrily use Shannon entropy to run ecosystem simulations.

Baez says there is generally no reason to suppose that biodiversity-related Shannon entropy reaches a maximum, but there are interesting cases where this is true. Namely, when there is a dominant, mutually interdependent cluster of species in the whole array of initial species.

 

[Genady]

29 minutes ago, joigus said:

Yeah, that's a beautiful idea, and I think it works. You can sub-divide also reversible processes into infinitesimally small irreversible sub-processes, and prove it for them too. I think it was very soon that people realised that entropy must be a state function, and the arguments were very similar.

I tend to look at these things more from the point of view of statistical mechanics. Boltzmann's H theorem has suffered waves of criticism through the years, but it seems to me to be quite robust.

https://en.wikipedia.org/wiki/H-theorem#Criticism_and_exceptions

I wouldn't call these criticisms "moot points," but I think all of them rest on some kind of oversimplification of physical systems.

Note that some members may find interesting: I've been watching a lecture by John Baez dealing with Shannon entropy, and the second principle of thermodynamics in the context of studying biodiversity.

It seems that people merrily use Shannon entropy to run ecosystem simulations.

Baez says there is generally no reason to suppose that biodiversity-related Shannon entropy reaches a maximum, but there are interesting cases where this is true. Namely, when there is a dominant, mutually interdependent cluster of species in the whole array of initial species.

 

Thank you. I'm glad you like the idea.

OTOH, I'm sorry, but I can't watch a 35 min video because of my APD. If it exists in writing somewhere...

[StringJunky]

40 minutes ago, Genady said:

Thank you. I'm glad you like the idea.

OTOH, I'm sorry, but I can't watch a 35 min video because of my APD. If it exists in writing somewhere...

Here's the slides: https://math.ucr.edu/home/baez/biodiversity/bio.pdf

[Genady]

46 minutes ago, StringJunky said:

Here's the slides: https://math.ucr.edu/home/baez/biodiversity/bio.pdf

Thank you! I'm glad I didn't need to watch that video. This was easy 😉

[joigus]

53 minutes ago, StringJunky said:

Here's the slides: https://math.ucr.edu/home/baez/biodiversity/bio.pdf

Thanks!!!

These notes are quite more complete than the video.  

[StringJunky]

My pleasure.  I don't know much but I know how to find things.

Edited March 29, 2022 by StringJunky