Central Limit Theorem

Theorem.

Let $X_1,\dots ,X_n$ be i.i.d. random variables with finite mean $\mu$. Then, the (weak) Law of Large Numbers says that for any fixed $ϵ>0$, $P(|\overline{X_n}-\mu |>ϵ)$ goes to 0 as $n\to \infty$. The Central Limit Theorem says that if $X_i$ have finite variance ${\sigma }^2$, then $\sqrt{n}(\overline{X_n}-\mu )\to N(0,{\sigma }^2)$ where $\sqrt{n}(\overline{X_n}-\mu )=\frac{1}{\sqrt{n}}{\sum }_{i=1}^n(X_i-\mu )$.

Remark.

$N(0,{\sigma }^2)$ maximizes the entropy among all distributions with variance ${\sigma }^2$.