Which of the Following Is a Biased Estimator? Understanding Bias in Statistical Estimation
When learning statistics, one of the first concepts that can feel a bit tricky is the difference between a biased estimator and an unbiased estimator. Consider this: in this article we will explore what makes an estimator biased, walk through common examples, and identify which typical estimator from a standard list is biased. In simple terms, an estimator is a rule or formula that uses sample data to infer a population parameter. If the rule consistently over‑ or under‑estimates that parameter, the estimator is said to be biased. By the end, you will have a clear picture of why bias matters and how to recognize it in practice Still holds up..
What Is a Biased Estimator?
In statistical theory, the bias of an estimator (\hat{\theta}) for a parameter (\theta) is defined as the difference between the expected value of the estimator and the true parameter value:
[ \text{Bias}(\hat{\theta}) = E[\hat{\theta}] - \theta ]
If this difference is zero, the estimator is unbiased; if it is non‑zero, the estimator is biased. e.Bias does not refer to a single “wrong” number but to a systematic error that persists across repeated sampling. Practically speaking, even a biased estimator can still be useful if it is consistent (i. , it converges to the true value as sample size grows) or if it has a lower mean‑squared error overall Turns out it matters..
Key Points to Remember
- Zero bias → unbiased estimator.
- Non‑zero bias → biased estimator.
- Bias is about expectation, not about any single estimate.
- A biased estimator may still be preferred in some contexts (e.g., lower variance).
Unbiased vs. Biased: Why It Matters
Understanding bias helps you choose the right tool for the job. In many introductory statistics courses, the most common estimators are:
- Sample mean ((\bar{x})) – used to estimate the population mean (\mu).
- Sample variance with (n) denominator – used to estimate (\sigma^2).
- Sample variance with (n-1) denominator – often taught as the unbiased estimator of (\sigma^2).
- Maximum likelihood estimator (MLE) of variance – derived from the normal distribution.
Each of these has a different bias profile. Let’s examine them in detail.
Common Estimators and Their Bias Status
1. Sample Mean ((\bar{x}))
The sample mean is perhaps the most intuitive estimator for the population mean. Its expectation is exactly the population mean:
[ E[\bar{x}] = \mu ]
That's why, the sample mean is unbiased. It also enjoys desirable properties such as consistency and efficiency under many conditions.
2. Sample Variance with (n) Denominator
When you calculate variance using the formula:
[ s_n^2 = \frac{1}{n}\sum_{i=1}^{n}(x_i - \bar{x})^2 ]
you are implicitly dividing by the sample size (n). In practice, this version is biased for the population variance (\sigma^2). The reason is that (\bar{x}) is itself estimated from the same data, which tends to make the squared deviations slightly smaller on average.
[ E[s_n^2] = \frac{n-1}{n}\sigma^2 ]
Thus, (s_n^2) underestimates (\sigma^2) by a factor of (\frac{n-1}{n}) That's the whole idea..
3. Sample Variance with (n-1) Denominator (Unbiased Estimator)
The classic correction, introduced by C. F. In real terms, gauss and later popularized by *R. A.
[ s^2 = \frac{1}{n-1}\sum_{i=1}^{n}(x_i - \bar{x})^2 ]
Because (E[s^2] = \sigma^2), this estimator is unbiased. It is the most commonly taught formula for sample variance in introductory courses But it adds up..
4. Maximum Likelihood Estimator (MLE) of Variance
When you derive the MLE for the variance of a normal distribution, you again divide by (n):
[ \hat{\sigma}^2_{\text{MLE}} = \frac{1}{n}\sum_{i=1}^{n}(x_i - \bar{x})^2 ]
Since the MLE coincides with (s_n^2), it inherits the same bias. That said, the MLE is consistent and has a smaller mean‑squared error for finite samples compared to the unbiased estimator. This trade‑off is a classic example of why bias alone does not dictate an estimator’s usefulness Nothing fancy..
Which of the Following Is a Biased Estimator? The Answer
If you were presented with a multiple‑choice list such as:
- A. Sample mean
- B. Sample variance using (n-1) denominator
- C. Sample variance using (n) denominator
- D. Unbiased estimator of population proportion
The biased estimator among these is C – the sample variance using (n) denominator. But it systematically underestimates the true population variance, especially in small samples. All other options are unbiased (or at least unbiased in expectation) Less friction, more output..
Scientific Explanation: Why Dividing by (n) Introduces Bias
The intuition behind the bias lies in the concept of degrees of freedom. When you compute the sample mean (\bar{x}), you have already used one piece of information from the data. The deviations ((x_i - \bar{x})) are therefore constrained: they must sum to zero. Plus, this constraint reduces the effective number of independent pieces of information by one. Dividing by (n) treats the data as if all (n) deviations were independent, which leads to an underestimate of variability Easy to understand, harder to ignore..
Mathematically, the expected sum of squared deviations is:
[ E\left[\sum_{i=1}^{n}(x_i - \bar{x})^2\right] = (n-1)\sigma^2 ]
Hence, dividing by (n) yields:
[ E[s_n^2] = \frac{n-1}{n}\sigma^2 ]
The factor (\frac{n-1}{n}) is precisely the bias. As (n) grows, the bias shrinks, which is why the estimator is asymptotically unbiased Simple, but easy to overlook..
Practical Implications
- Small Samples: The bias is most pronounced when (n) is small. As an example, with (n=5), the biased variance under
…underestimates the true variance by roughly 20 % when only five observations are available, because (\frac{n-1}{n}=0.8). Which means with (n=10) the shrinkage falls to 10 %, and by the time (n=30) the bias is under 3. 5 %. In many applied settings—especially those involving modest sample sizes such as pilot studies, early‑phase clinical trials, or rare‑event surveys—this systematic under‑dispersion can lead to overly narrow confidence intervals, inflated type‑I error rates, and misleading power calculations.
Beyond the variance estimator, other familiar statistics exhibit similar bias when the denominator is chosen without accounting for lost degrees of freedom. Here's a good example: the naïve estimator of the population standard deviation, (\sqrt{\frac{1}{n}\sum (x_i-\bar{x})^2}), is also biased downward, although the bias is less severe because the square‑root transformation attenuates the effect. Similarly, the sample covariance computed with (n) in the denominator underestimates the true covariance, whereas the unbiased version uses (n-1).
Practically, the choice between the biased (MLE) and unbiased versions hinges on the goal of the analysis:
- Inference‑focused work (hypothesis testing, confidence‑interval construction) benefits from the unbiased estimator because it yields correct expected coverage probabilities under repeated sampling.
- Prediction‑oriented or machine‑learning contexts often prefer the MLE version; its slightly smaller mean‑squared error can improve out‑of‑sample performance, especially when the model is regularized or when the variance estimate is plugged into a larger optimization routine.
- Bayesian approaches sidestep the issue entirely by placing a prior on (\sigma^2) and deriving a posterior estimate that naturally balances bias and variance.
Simply put, dividing by (n) when computing a sample variance introduces a predictable, downward bias that disappears only asymptotically. Recognizing the source of this bias— the loss of one degree of freedom from estimating the mean—helps practitioners decide whether to adopt the unbiased (n-1) divisor for rigorous inference or to accept the modest bias of the MLE version when predictive accuracy or computational convenience is key. The key is to match the estimator’s properties to the specific objectives of the analysis rather than to treat bias as an absolute vice Small thing, real impact..
