Entropy of Mixing

[sethoflagos]

My scribblings came up with the following interesting function

f(x1,x2,... xn) = -ln(x1^x1*x2^x2*,... xn^xn)

Where:

x1+x2+... xn = 1

0<xi<1

I suspect that this function ranges 0 to ln(n) but a proof is beyond me.

Assistance would be most appreciated.

 

[Genady]

4 hours ago, sethoflagos said:

My scribblings came up with the following interesting function

f(x1,x2,... xn) = -ln(x1^x1*x2^x2*,... xn^xn)

Where:

x1+x2+... xn = 1

0<xi<1

I suspect that this function ranges 0 to ln(n) but a proof is beyond me.

Assistance would be most appreciated.

 

It gets a min when one xi=1 and all others are 0. It gets a max when all xi are equal.

[sethoflagos]

13 minutes ago, Genady said:

It gets a min when one xi=1 and all others are 0. It gets a max when all xi are equal.

Yes, thank you, Genady. Trial and error seems to indicate so.

But how does one derive those values mathematically?

[Genady]

9 minutes ago, sethoflagos said:

Yes, thank you, Genady. Trial and error seems to indicate so.

But how does one derive those values mathematically?

I'd start with simplifying the expression and then finding an extremum using Lagrange multiplier method.

[Genady]

The approach above should give you the max(f())=ln(n) quite fast. To get min(f), assume that ln(xj) is the smallest of all ln(xi), take the condition

x1+x2+... xn = 1,

multiply it by -ln(xj):

-(x1+x2+... xn)*ln(xj)=-ln(xj).

Because all ln(xi)>=ln(xj),

-(x1*ln(x1)+...+xn*ln(xn))>=-ln(xj).

The left side is -ln(x1^x1*x2^x2*... xn^xn), the right side -ln(xj)>=0. Thus

-ln(x1^x1*x2^x2*... xn^xn)>=0.

And we know how to get it =0. Thus 0 is the minimum.

[sethoflagos]

On 4/3/2022 at 11:53 PM, Genady said:

To get min(f), assume that ln(xj) is the smallest of all ln(xi), take the condition

x1+x2+... xn = 1,

multiply it by -ln(xj):

-(x1+x2+... xn)*ln(xj)=-ln(xj).

Because all ln(xi)>=ln(xj),

-(x1*ln(x1)+...+xn*ln(xn))>=-ln(xj).

The left side is -ln(x1^x1*x2^x2*... xn^xn), the right side -ln(xj)>=0. Thus

-ln(x1^x1*x2^x2*... xn^xn)>=0.

And we know how to get it =0. Thus 0 is the minimum.

I'm so sorry, Genady, I missed this last post of yours.

So for the maximum we get the Legrangian function

L(x1, x2 ... xn, k) = -x1.ln(x1) -x2.ln(x2) ... -xn.ln(xn) - k.(x1 +x2 ... +xn -1)

The partial derivatives for all xi = -(ln(xi) +k +1) and therefore are zero at -ln(xi) = k+1  or xi = e^(-k-1)

The partial derivative wrt k is zero for the constraint condition x1 +x2 ... +xn = 1

Hence n.e^(-k-1) = 1 which solves to k = ln(n) -1

Hence xi = 1/n and ln(xi) = -ln(n)

Hence max value of original function = n.(1/n.ln(n)) = ln(n)

On 4/3/2022 at 11:53 PM, Genady said:

The approach above should give you the max(f())=ln(n) quite fast.

And you were not wrong! Thankyou so much for putting a new string on my mathematical bow.

I think I'll go away now and quietly try this method out on a few other functions.