Parametric model

In statistics, a parametric model or parametric family or finite-dimensional model is a family of distributions that can be described using a finite number of parameters. These parameters are usually collected together to form a single k-dimensional parameter vector θ = (θ₁, θ₂, …, θ_k).

Parametric models are contrasted with the semi-parametric, semi-nonparametric, and non-parametric models, all of which consist of an infinite set of "parameters" for description. The distinction between these four classes is as follows:

in a "parametric" model all the parameters are in finite-dimensional parameter spaces;
a model is "non-parametric" if all the parameters are in infinite-dimensional parameter spaces;
a "semi-parametric" model contains finite-dimensional parameters of interest and infinite-dimensional nuisance parameters;
a "semi-nonparametric" model has both finite-dimensional and infinite-dimensional unknown parameters of interest.

Some statisticians believe that the concepts "parametric", "non-parametric", and "semi-parametric" are ambiguous.^[1] It can also be noted that the set of all probability measures has cardinality of continuum, and therefore it is possible to parametrize any model at all by a single number in (0,1) interval.^[2] This difficulty can be avoided by considering only "smooth" parametric models.

Definition

A parametric model is a collection of probability distributions such that each member of this collection, P_θ, is described by a finite-dimensional parameter θ. The set of all allowable values for the parameter is denoted Θ ⊆ R^k, and the model itself is written as

\mathcal{P} = \big\{ P_\theta\ \big|\ \theta\in\Theta \big\}.

When the model consists of absolutely continuous distributions, it is often specified in terms of corresponding probability density functions:

\mathcal{P} = \big\{ f_\theta\ \big|\ \theta\in\Theta \big\}.

The parametric model is called identifiable if the mapping θ ↦ P_θ is invertible, that is there are no two different parameter values θ₁ and θ₂ such that P_θ₁ = P_θ₂.

Examples

The Poisson family of distributions is parametrized by a single number λ > 0:
$\mathcal{P} = \Big\{\ p_\lambda(j) = \tfrac{\lambda^j}{j!}e^{-\lambda},\ j=0,1,2,3,\dots \ \Big|\ \lambda>0 \ \Big\},$

where p_λ is the probability mass function. This family is an exponential family.
The normal family is parametrized by θ = (μ,σ), where μ ∈ R is a location parameter, and σ > 0 is a scale parameter. This parametrized family is both an exponential family and a location-scale family:
$\mathcal{P} = \Big\{\ f_\theta(x) = \tfrac{1}{\sqrt{2\pi}\sigma} e^{ -\frac{1}{2\sigma^2}(x-\mu)^2 }\ \Big|\ \mu\in\mathbb{R}, \sigma>0 \ \Big\}.$
The Weibull translation model has three parameters θ = (λ, β, μ):
$\mathcal{P} = \Big\{\ f_\theta(x) = \tfrac{\beta}{\lambda} \left(\tfrac{x-\mu}{\lambda}\right)^{\beta-1}\! \exp\!\big(\!-\!\big(\tfrac{x-\mu}{\lambda}\big)^\beta \big)\, \mathbf{1}_{\{x>\mu\}} \ \Big|\ \lambda>0,\, \beta>0,\, \mu\in\mathbb{R} \ \Big\}.$

This model is not regular (see definition below) unless we restrict β to lie in the interval (2, +∞).

Regular parametric model

Let $\mu$ be a fixed σ-finite measure on a measurable space $(\Omega ,{\mathcal {F}})$ , and $\scriptstyle \mathcal{M}_\mu$ the collection of all probability measures dominated by $\mu$ . Then we will call $\mathcal{P}\!=\!\{ P_\theta|\, \theta\in\Theta \} \subseteq \mathcal{M}_\mu$ a regular parametric model if the following requirements are met:^[3]

$\Theta$ is an open subset of $\mathbb {R} ^{k}$ .
The map
$\theta\mapsto s(\theta)=\sqrt{dP_\theta/d\mu}$

from $\Theta$ to $L^{2}(\mu )$ is Fréchet differentiable: there exists a vector $\dot{s}(\theta) = (\dot{s}_1(\theta),\,\ldots,\,\dot{s}_k(\theta))$ such that

$\lVert s(\theta +h)-s(\theta )-{\dot {s}}(\theta )'h\rVert =o(|h|){\text{ as }}h\to 0,$

where ′ denotes matrix transpose.
The map $\theta\mapsto\dot{s}(\theta)$ (defined above) is continuous on $\Theta$ .
The $k\times k$ Fisher information matrix
$I(\theta) = 4\int \dot{s}(\theta)\dot{s}(\theta)'d\mu$

is non-singular.

Properties

Sufficient conditions for regularity of a parametric model in terms of ordinary differentiability of the density function ƒ_θ are following:^[4]
1. The density function ƒ_θ(x) is continuously differentiable in θ for μ-almost all $x$ , with gradient $\nabla f_{\theta }$ .
2. The score function
  $z_\theta = \frac{\nabla f_\theta}{f_\theta} \cdot \mathbf{1}_{\{f_\theta>0\}}$
  
  belongs to the space $L^{2}(P_{\theta })$ of square-integrable functions with respect to the measure $P_{\theta }$ .
3. The Fisher information matrix I(θ), defined as
  $I_\theta = \int \!z_\theta z_\theta' \,dP_\theta$
  
  is nonsingular and continuous in θ.
If conditions (i)−(iii) hold then the parametric model is regular.
Local asymptotic normality.
If the regular parametric model is identifiable then there exists a uniformly $\scriptstyle \sqrt{n}$ -consistent and efficient estimator of its parameter θ.^[5]

Notes

↑ Le Cam & Yang 2000, §7.4
↑ Bickel et al. 1998, p. 2
↑ Bickel et al. 1998, p. 12
↑ Bickel et al. 1998, p.13, prop.2.1.1
↑ Bickel et al. 1998, Theorems 2.5.1, 2.5.2

Bibliography

Bickel, Peter J.; Doksum, Kjell A. (2001). Mathematical Statistics: Basic and Selected Topics. Volume 1 (Second (updated printing 2007) ed.). Prentice-Hall.
Bickel, Peter J.; Klaassen, Chris A. J.; Ritov, Ya’acov; Wellner, Jon A. (1998). Efficient and Adaptive Estimation for Semiparametric Models. Springer.
Davidson, A. C. (2003). Statistical Models. Cambridge University Press.
Freedman, David A. (2009). Statistical Models: Theory and Practice (Second ed.). Cambridge University Press. ISBN 978-0-521-67105-7.
Le Cam, Lucien; Yang, Grace Lo (2000). Asymptotics in Statistics: some basic concepts. Springer.
Lehmann, Erich L.; Casella, George (1998). Theory of Point Estimation (2nd ed.). Springer.
Lehmann, Erich L.; Romano, Joseph P. (2005). Testing Statistical Hypotheses (3rd ed.). Springer.
Liese, Friedrich; Miescke, Klaus-J. (2008). Statistical Decision Theory: Estimation, Testing, and Selection. Springer.
Pfanzagl, Johann; with the assistance of R. Hamböker (1994). Parametric Statistical Theory. Walter de Gruyter. MR 1291393.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.

[1] Le Cam & Yang 2000, §7.4

[2] Bickel et al. 1998, p. 2

[3] Bickel et al. 1998, p. 12

[4] Bickel et al. 1998, p.13, prop.2.1.1

[5] Bickel et al. 1998, Theorems 2.5.1, 2.5.2