Estimators

Definition.

Given random variables $X_1,\dots ,X_n$ a statistic is $W=f(X_1,\dots ,X_n)$ for any function $f$.

Definition.

An estimator (of $\theta$) is a statistic (for inference).

Definition.

The bias of an estimator $W$ for $\theta$ is

$$Bia{s}_{\theta }(W):={\mathbb{E}}_{\theta }[W]-\theta .$$

We say $W$ is unbiased if $Bia{s}_{\theta }(W)=0$.

---

The naive sample variance $\frac{1}{n}{\sum }_{i=1}^n(X_i-\bar{X}{)}^2$ where $\bar{X}$ is the sample mean is a biased estimator of the real variance since the expected value of this naive sample variance is $\frac{n-1}{n}{\sigma }^2$. The actual sample variance which is an unbiased estimator of the real variance is $\frac{1}{n-1}{\sum }_{i=1}^n(X_i-\bar{X}{)}^2$.