 Consistent estimator

In statistics, a sequence of estimators for parameter θ_{0} is said to be consistent (or asymptotically consistent) if this sequence converges in probability to θ_{0}. It means that the distributions of the estimators become more and more concentrated near the true value of the parameter being estimated, so that the probability of the estimator being arbitrarily close to θ_{0} converges to one.
In practice one usually constructs a single estimator as a function of an available sample of size n, and then imagines being able to keep collecting data and expanding the sample ad infinitum. In this way one would obtain a sequence of estimators indexed by n and the notion of consistency will be understood as the sample size “grows to infinity”. If this sequence converges in probability to the true value θ_{0}, we call it a consistent estimator; otherwise the estimator is said to be inconsistent.
The consistency as defined here is sometimes referred to as the weak consistency. When we replace the convergence in probability with the almost sure convergence, then the sequence of estimators is said to be strongly consistent.
Contents
Definition
Loosely speaking, an estimator T_{n} of parameter θ is said to be consistent, if it converges in probability to the true value of the parameter:^{[1]}
A more rigorous definition takes into account the fact that θ is actually unknown, and thus the convergence in probability must take place for every possible value of this parameter. Suppose {p_{θ}: θ ∈ Θ} is a family of distributions (the parametric model), and X^{θ} = {X_{1}, X_{2}, … : X_{i} ~ p_{θ}} is an infinite sample from the distribution p_{θ}. Let { T_{n}(X^{θ}) } be a sequence of estimators for some parameter g(θ). Usually T_{n} will be based on the first n observations of a sample. Then this sequence {T_{n}} is said to be (weakly) consistent if ^{[2]}
This definition uses g(θ) instead of simply θ, because often one is interested in estimating a certain function or a subvector of the underlying parameter. In the next example we estimate the location parameter of the model, but not the scale:
Example: sample mean for normal random variables
Suppose one has a sequence of observations {X_{1}, X_{2}, …} from a normal N(μ, σ^{2}) distribution. To estimate μ based on the first n observations, we use the sample mean: T_{n} = (X_{1} + … + X_{n})/n. This defines a sequence of estimators, indexed by the sample size n.
From the properties of the normal distribution, we know that T_{n} is itself normally distributed, with mean μ and variance σ^{2}/n. Equivalently, has a standard normal distribution. Then
as n tends to infinity, for any fixed ε > 0. Therefore, the sequence T_{n} of sample means is consistent for the population mean μ.
Establishing consistency
The notion of asymptotic consistency is very close, almost synonymous to the notion of convergence in probability. As such, any theorem, lemma, or property which establishes convergence in probability may be used to prove the consistency. Many such tools exist:
 In order to demonstrate consistency directly from the definition one can use the inequality ^{[3]}
the most common choice for function h being either the absolute value (in which case it is known as Markov inequality), or the quadratic function (respectively Chebychev's inequality).
 Another useful result is the continuous mapping theorem: if T_{n} is consistent for θ and g(·) is a realvalued function continuous at point θ, then g(T_{n}) will be consistent for g(θ):^{[4]}
 Slutsky’s theorem can be used to combine several different estimators, or an estimator with a nonrandom covergent sequence. If T_{n} →^{p}α, and S_{n} →^{p}β, then ^{[5]}
 If estimator T_{n} is given by an explicit formula, then most likely the formula will employ sums of random variables, and then the law of large numbers can be used: for a sequence {X_{n}} of random variables and under suitable conditions,
 If estimator T_{n} is defined implicitly, for example as a value that maximizes certain objective function (see extremum estimator), then a more complicated argument involving stochastic equicontinuity has to be used.^{[6]}
Bias versus consistency
Unbiased but not consistent
An estimator can be unbiased but not consistent. For example, for an iid sample {x
1,..., x
n} one can use T(X) = x
1 as the estimator of the mean E[x]. This estimator is obviously unbiased, and obviously inconsistent.Biased but consistent
Alternatively, an estimator can be biased but consistent. For example if the mean is estimated by it is biased, but as , it approaches the correct value, and so it is consistent.
See also
 Fisher consistency — alternative, although rarely used concept of consistency for the estimators
 Consistent test — the notion of consistency in the context of hypothesis testing
Notes
 ^ Amemiya 1985, Definition 3.4.2
 ^ Lehman & Casella 1998, p. 332
 ^ Amemiya 1985, equation (3.2.5)
 ^ Amemiya 1985, Theorem 3.2.6
 ^ Amemiya 1985, Theorem 3.2.7
 ^ Newey & McFadden (1994, Chapter 2)
References
 Amemiya, Takeshi (1985). Advanced econometrics. Harvard University Press. ISBN 0674005600.
 Lehmann, E. L.; Casella, G. (1998). Theory of Point Estimation (2nd ed.). Springer. ISBN 0387985026.
 Newey, W.; McFadden, D. (1994). Large sample estimation and hypothesis testing. In “Handbook of Econometrics”, Vol. 4, Ch. 36. Elsevier Science. ISBN 0444887660.
 Nikulin, M.S. (2001), "Consistent estimator", in Hazewinkel, Michiel, Encyclopaedia of Mathematics, Springer, ISBN 9781556080104, http://eom.springer.de/C/c025240.htm
Categories: Statistical theory
 Statistical inference
 Estimation theory
 Asymptotic statistical theory
Wikimedia Foundation. 2010.
Look at other dictionaries:
Estimator — In statistics, an estimator is a function of the observable sample data that is used to estimate an unknown population parameter (which is called the estimand ); an estimate is the result from the actual application of the function to a… … Wikipedia
consistent — adjective Etymology: Latin consistent , consistens, present participle of consistere Date: 1638 1. archaic possessing firmness or coherence 2. a. marked by harmony, regularity, or steady continuity … New Collegiate Dictionary
Newey–West estimator — A Newey–West estimator is used in statistics and econometrics to provide an estimate of the covariance matrix of the parameters of a regression type model when this model is applied in situations where the standard assumptions of regression… … Wikipedia
Heteroscedasticityconsistent standard errors — In statistics, a frequent assumption in linear regression is that the disturbances u i have the same variance. When this is not the case, we get heteroskedasticity in the estimated residuals scriptstylewidehat{u i} . Heteroskedasticity consistent … Wikipedia
Errorsinvariables models — In statistics and econometrics, errors in variables models or measurement errors models are regression models that account for measurement errors in the independent variables. In contrast, standard regression models assume that those regressors… … Wikipedia
Instrumental variable — In statistics, econometrics, and related disciplines, the method of instrumental variables (IV) is used to estimate causal relationships when controlled experiments are not feasible. Statistically, IV methods allow consistent estimation when the… … Wikipedia
List of statistics topics — Please add any Wikipedia articles related to statistics that are not already on this list.The Related changes link in the margin of this page (below search) leads to a list of the most recent changes to the articles listed below. To see the most… … Wikipedia
Scale parameter — In probability theory and statistics, a scale parameter is a special kind of numerical parameter of a parametric family of probability distributions. The larger the scale parameter, the more spread out the distribution.DefinitionIf a family of… … Wikipedia
Consistency (statistics) — In statistics, consistency of procedures such as confidence intervals or hypothesis tests involves their behaviour as the number of items in the data set to which they are applied increases indefinitely. In particular, consistency requires that… … Wikipedia
Mean difference — The mean difference is a measure of statistical dispersion equal to the average absolute difference of two independent values drawn from a probability distribution. A related statistic is the relative mean difference, which is the mean difference … Wikipedia