Related Links (Outside this Site)

---

(2006-09-29)     The method of Lagrange multipliers
Maximizing under one constraint, or several constraints.

Consider a smooth enough function  S  of  W  variables:  S ( x₁ , x₂ , ... , x_W ).

We seek to maximize  S  subject to the  constraint  that some other function  F  of those same variables is a given constant.  Lagrange's method associates a parameter  l  to such a constraint and introduces a new function  L :

L   =   S  +  l F

The key observation is that the constrained maximum we seek  (assuming there is one)  occurs at a  saddlepoint of  L  (i.e., dL = 0)  for a specific value of  l.

Proof:   At the constrained maximum, any displacement which maintains the constraint entails a vanishing variation of  S   (i.e.,  dF = 0  Þ  dS = 0).

" dx1 , ... , dxW   {  å i	¶ F	dxi   = 0  }   Þ   {  å i	¶ S	dxi   = 0  }
	¶xi		¶xi

In other words, any  W-dimensional vector which is perpendicular to  [¶F/¶x_i]  is also perpendicular to  [¶S/¶x_i].  Therefore, these two are proportional:

$ l ,   " i ,	¶ S	+   l	¶ F	=   0
	¶xi		¶xi

The parameter  l  thus obtained is called a  Lagrange multiplier.  One such  Lagrange multiplier  corresponds to each of  several  simultaneous constraints.  Any constrained  saddlepoint  (possibly a maximum)  of  S  is then an unrestricted saddlepoint of the following function  L ,  and  vice-versa.

L   =   S  +  å_n   l_n F_n

$ l1, l2 ...   " i ,	¶ S	+   ån   ln	¶ F	=   0
	¶xi		¶xi

The (constant) value of each  F_n  can be retrieved as   ¶L / ¶l_n.

---

(2006-09-29)     Micro-Canonical Distribution
For an isolated system, entropy is maximal with equiprobable states.

Let's apply the above to Claude Shannon's definition of statistical entropy in terms of the respective probabilities of the  W  possible states of the system:

S ( p1 , p2 , ... , pW )   =

W å n = 1

- k  pn  Log (pn )

The basic constraint for probabilities is  completeness :  p₁ + p₂ + ... + p_W  =  1.  This is the  only  constraint for a completely  isolated  system.

L   =   S + l F   =   S + l ( p₁ + p₂ + ... + p_W )
 
0   =   ¶L / ¶p_i   =   l  -  k [ 1 + Log(p_i ) ]

Therefore, all values of  p_i  are equal to  exp( ^l/_k-1)   =   ¹/_W .  Plugging this equiprobability into the above expression of  S,  we obtain Boltzmann's relation:

S   =   k  Log(W)

Classical equipartition of energy :

---

(2006-09-29)     Canonical Distribution
In a heat bath, probabilities are proportional to  Boltzmann factors.

Let  E_i  be the energy of state i.  Putting the system in thermal equilibrium with a "heat bath" makes its  average  energy  å p_i E_i  constant.  This can be viewed as an additional "constraint" corresponding to a new Lagrange multiplier b.

L   =   S  +  l å p_i   +  b å p_i E_i

b  turns out to be inversely proportional to the temperature of the bath.

Canonical: Average energy å p_i E_i is constant for the system in contact with a heat bath. Lagrange multiplier is inversely proportional to temperature.

Micro-canonical: Given energy for the system... Special case is equipartion of energy between loosely connected degrees of freedom.

---

(2006-09-29)     Grand-Canonical Distribution
Taking into account the possibility of chemical exchanges.

---

(2006-09-29)     Bose-Einstein Statistics
Many particles (bosons) may occupy the same state.

---

(2006-09-29)     Fermi-Dirac Statistics
All particles (fermions) are in different states.

---

(2006-09-29)     Boltzmann's Statistics   (for either bosons or fermions)
The low occupancy limit where almost all states are unoccupied.

---

(2006-09-30)     Maxwell-Boltzmann distribution of speeds
Boltzmann statistics applied to the molecules in a classical perfect gas.

---

(2006-09-29)     Partition Function
Thermal summary of a distribution.