Statistical Mechanics Connection

Central Question

Why do information theory and thermodynamics use closely related entropy expressions?

Why This Matters

The relationship between distributions, coarse descriptions and physical equilibrium explains both the common mathematics and the different interpretations of entropy.

Learning Prompt

I know probability and undergraduate mechanics. Explore the connection between Shannon entropy, Gibbs entropy and Boltzmann's counting formula through a simple finite model of microstates and macrostates. Derive how $-k_B\sum_i p_i\ln p_i$ reduces to $k_B\ln\Omega$ for equally likely accessible microstates, then explain what physical assumptions make such expressions thermodynamic entropies. Distinguish counting, uncertainty and coarse-graining; do not equate information entropy with heat without qualification. Investigate maximum entropy under constraints, equilibrium and the puzzle of reversible microscopic laws versus macroscopic irreversibility. Ask me to identify where assumptions enter.

Ideas

Examples

Questions

Connections

• Law of Large Numbers
• Markov Chains
• What Is Entropy?
• Shannon Information
• Geometry Meets Physics
• Week overview

One Thing That Surprised Me

Further Rabbit Holes