Función de varianza

Este artículo necesita citas adicionales para su verificación . Por favor, ayuda a mejorar este artículo mediante la adición de citas de fuentes confiables . El material no obtenido puede ser cuestionado y eliminado. Buscar fuentes:  "Función de variación"  -  noticias  · periódicos  · libros  · académico  · JSTOR ( marzo de 2014 ) ( Aprenda cómo y cuándo eliminar este mensaje de plantilla )

Para conocer la variación en función de la separación espacio-temporal, consulte Variograma .

Parte de una serie sobre
Análisis de regresión
Modelos
Regresión lineal Regresión simple Regresión polinomial Modelo linear general
Modelo lineal generalizado Elección discreta Regresión binomial Regresión binaria Regresión logística Logit multinomial Logit mixto Probit Probit multinomial Logit ordenado Probit ordenado Poisson
Modelo multinivel Efectos fijos Efectos aleatorios Modelo lineal de efectos mixtos Modelo no lineal de efectos mixtos
Regresión no lineal No paramétrico Semiparamétrico Robusto Cuantil Isotónico Componentes principales Menor ángulo Local Segmentario
Errores en variables
Estimacion
Mínimos cuadrados Lineal No lineal
Ordinario Ponderado Generalizado
Parcial Total No negativo Regresión de crestas Regularizado
Desviaciones mínimas absolutas Reponderado iterativamente Bayesiano Multivariado bayesiano
Fondo
Validación de regresión Respuesta media y prevista Errores y residuales Bondad de ajuste Residuo estudentizado Teorema de Gauss-Markov
Portal de matemáticas
v t mi

En estadística , la función de varianza es una función uniforme que describe la varianza de una cantidad aleatoria en función de su media . La función de varianza es una medida de heterocedasticidad y juega un papel importante en muchos entornos de modelado estadístico. Es un ingrediente principal en el marco del modelo lineal generalizado y una herramienta utilizada en regresión no paramétrica , ^[1] regresión semiparamétrica ^[1] y análisis de datos funcionales . ^[2]En el modelado paramétrico, las funciones de varianza adoptan una forma paramétrica y describen explícitamente la relación entre la varianza y la media de una cantidad aleatoria. En un entorno no paramétrico, se supone que la función de varianza es una función suave .

Intuición

En la configuración de un modelo de regresión, el objetivo es establecer si existe o no una relación entre una variable de respuesta y un conjunto de variables predictoras. Además, si existe una relación, el objetivo es poder describir esta relación de la mejor manera posible. Un supuesto principal en la regresión lineal es la varianza constante u (homocedasticidad), lo que significa que diferentes variables de respuesta tienen la misma varianza en sus errores, en cada nivel de predictor. Esta suposición funciona bien cuando la variable de respuesta y la variable predictora son conjuntamente Normal, consulte Distribución normal . Como veremos más adelante, la función de varianza en el entorno Normal es constante, sin embargo, debemos encontrar una manera de cuantificar la heterocedasticidad (varianza no constante) en ausencia de normalidad conjunta.

Cuando es probable que la respuesta siga una distribución que es un miembro de la familia exponencial, un modelo lineal generalizado puede ser más apropiado para usar y, además, cuando no deseamos forzar un modelo paramétrico en nuestros datos, un modelo no paramétrico El enfoque de regresión puede ser útil. La importancia de poder modelar la varianza en función de la media radica en una mejor inferencia (en un entorno paramétrico) y en la estimación de la función de regresión en general, para cualquier entorno.

Las funciones de varianza juegan un papel muy importante en la estimación e inferencia de parámetros. En general, la estimación de máxima verosimilitud requiere que se defina una función de verosimilitud. Este requisito implica entonces que primero se debe especificar la distribución de las variables de respuesta observadas. Sin embargo, para definir una cuasi verosimilitud, solo es necesario especificar una relación entre la media y la varianza de las observaciones para luego poder usar la función de cuasi verosimilitud para la estimación. ^{[3] La} estimación de cuasi-verosimilitud es particularmente útil cuando hay sobredispersión . La sobredispersión ocurre cuando hay más variabilidad en los datos de la que debería esperarse según la distribución supuesta de los datos.

En resumen, para asegurar una inferencia eficiente de los parámetros de regresión y la función de regresión, se debe tener en cuenta la heterocedasticidad. Las funciones de varianza cuantifican la relación entre la varianza y la media de los datos observados y, por lo tanto, desempeñan un papel importante en la estimación y la inferencia de la regresión.

Tipos

La función de varianza y sus aplicaciones surgen en muchas áreas del análisis estadístico. Un uso muy importante de esta función es en el marco de modelos lineales generalizados y regresión no paramétrica .

Modelo lineal generalizado

Cuando se ha especificado un miembro de la familia exponencial , la función de varianza se puede derivar fácilmente. ^{[4] : 29} La forma general de la función de varianza se presenta en el contexto familiar exponencial, así como formas específicas para Normal, Bernoulli, Poisson y Gamma. Además, describimos las aplicaciones y el uso de las funciones de varianza en la estimación de máxima verosimilitud y la estimación de cuasi-verosimilitud.

Derivación

El modelo lineal generalizado (GLM) , es una generalización del análisis de regresión ordinario que se extiende a cualquier miembro de la familia exponencial . Es particularmente útil cuando la variable de respuesta es categórica, binaria o sujeta a una restricción (por ejemplo, solo las respuestas positivas tienen sentido). En esta página se resume un resumen rápido de los componentes de un GLM, pero para obtener más detalles e información, consulte la página sobre modelos lineales generalizados .

Un GLM consta de tres ingredientes principales:

1. Componente aleatorio: una distribución de y de la familia exponencial,${\displaystyle E[y\mid X]=\mu }$

2. Predictor lineal: ${\displaystyle \eta =XB=\sum _{j=1}^{p}X_{ij}^{T}B_{j}}$

3. Función de enlace: ${\displaystyle \eta =g(\mu ),\mu =g^{-1}(\eta )}$

Primero, es importante derivar un par de propiedades clave de la familia exponencial.

Cualquier variable aleatoria en la familia exponencial tiene una función de densidad de probabilidad de la forma,${\displaystyle {\textit {y}}}$

${\displaystyle f(y,\theta ,\phi )=\exp \left({\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )\right)}$

con loglikelihood,

${\displaystyle \ell (\theta ,y,\phi )=\log(f(y,\theta ,\phi ))={\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )}$

Aquí, está el parámetro canónico y el parámetro de interés, y es un parámetro de molestia que juega un papel en la varianza. Usamos las identidades de Bartlett para derivar una expresión general para la función de varianza . El primer y segundo resultado de Bartlett asegura que bajo condiciones adecuadas (ver la regla integral de Leibniz ), para una función de densidad dependiente de ,${\displaystyle \theta }$${\displaystyle \phi }$${\displaystyle \theta ,f_{\theta }()}$

${\displaystyle \operatorname {E} _{\theta }\left[{\frac {\partial }{\partial \theta }}\log(f_{\theta }(y))\right]=0}$

${\displaystyle \operatorname {Var} _{\theta }\left[{\frac {\partial }{\partial \theta }}\log(f_{\theta }(y))\right]+\operatorname {E} _{\theta }\left[{\frac {\partial ^{2}}{\partial \theta ^{2}}}\log(f_{\theta }(y))\right]=0}$

Estas identidades conducen a cálculos simples del valor esperado y la varianza de cualquier variable aleatoria en la familia exponencial .${\displaystyle {\textit {y}}}$${\displaystyle E_{\theta }[y],Var_{\theta }[y]}$

Valor esperado de Y : Tomando la primera derivada con respecto al logaritmo de la densidad en la forma de familia exponencial descrita anteriormente, tenemos${\displaystyle \theta }$

${\displaystyle {\frac {\partial }{\partial \theta }}\log(f(y,\theta ,\phi ))={\frac {\partial }{\partial \theta }}\left[{\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )\right]={\frac {y-b'(\theta )}{\phi }}}$

Luego, tomar el valor esperado y establecerlo igual a cero conduce a,

${\displaystyle \operatorname {E} _{\theta }\left[{\frac {y-b'(\theta )}{\phi }}\right]={\frac {\operatorname {E} _{\theta }[y]-b'(\theta )}{\phi }}=0}$

${\displaystyle \operatorname {E} _{\theta }[y]=b'(\theta )}$

Varianza de Y: Para calcular la varianza usamos la segunda identidad de Bartlett,

${\displaystyle \operatorname {Var} _{\theta }\left[{\frac {\partial }{\partial \theta }}\left({\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )\right)\right]+\operatorname {E} _{\theta }\left[{\frac {\partial ^{2}}{\partial \theta ^{2}}}\left({\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )\right)\right]=0}$

${\displaystyle \operatorname {Var} _{\theta }\left[{\frac {y-b'(\theta )}{\phi }}\right]+\operatorname {E} _{\theta }\left[{\frac {-b''(\theta )}{\phi }}\right]=0}$

${\displaystyle \operatorname {Var} _{\theta }\left[y\right]=b''(\theta )\phi }$

Ahora tenemos una relación entre y , a saber${\displaystyle \mu }$${\displaystyle \theta }$

${\displaystyle \mu =b'(\theta )}$y , que permite una relación entre y la varianza,${\displaystyle \theta =b'^{-1}(\mu )}$${\displaystyle \mu }$

${\displaystyle V(\theta )=b''(\theta )={\text{the part of the variance that depends on }}\theta }$

${\displaystyle \operatorname {V} (\mu )=b''(b'^{-1}(\mu )).\,}$

Tenga en cuenta que porque , entonces es invertible. Derivamos la función de varianza para algunas distribuciones comunes.${\displaystyle \operatorname {Var} _{\theta }\left[y\right]>0,b''(\theta )>0}$${\displaystyle b':\theta \rightarrow \mu }$

Ejemplo: normal

The Normal distribution is a special case where the variance function is a constant. Let ${\displaystyle y\sim N(\mu ,\sigma ^{2})}$ then we put the density function of y in the form of the exponential family described above:

${\displaystyle f(y)=\exp \left({\frac {y\mu -{\frac {\mu ^{2}}{2}}}{\sigma ^{2}}}-{\frac {y^{2}}{2\sigma ^{2}}}-{\frac {1}{2}}\ln {2\pi \sigma ^{2}}\right)}$

where

${\displaystyle \theta =\mu ,}$

${\displaystyle b(\theta )={\frac {\mu ^{2}}{2}},}$

${\displaystyle \phi =\sigma ^{2},}$

${\displaystyle c(y,\phi )=-{\frac {y^{2}}{2\sigma ^{2}}}-{\frac {1}{2}}\ln {2\pi \sigma ^{2}}}$

To calculate the variance function ${\displaystyle V(\mu )}$, we first express ${\displaystyle \theta }$ as a function of ${\displaystyle \mu }$. Then we transform ${\displaystyle V(\theta )}$ into a function of ${\displaystyle \mu }$

${\displaystyle \theta =\mu }$

${\displaystyle b'(\theta )=\theta =\operatorname {E} [y]=\mu }$

${\displaystyle V(\theta )=b''(\theta )=1}$

Therefore, the variance function is constant.

Example – Bernoulli

Let ${\displaystyle y\sim {\text{Bernoulli}}(p)}$, then we express the density of the Bernoulli distribution in exponential family form,

${\displaystyle f(y)=\exp \left(y\ln {\frac {p}{1-p}}+\ln(1-p)\right)}$

${\displaystyle \theta =\ln {\frac {p}{1-p}}=}$ logit(p), which gives us ${\displaystyle p={\frac {e^{\theta }}{1+e^{\theta }}}=}$ expit${\displaystyle (\theta )}$

${\displaystyle b(\theta )=\ln(1+e^{\theta })}$ and

${\displaystyle b'(\theta )={\frac {e^{\theta }}{1+e^{\theta }}}=}$ expit${\displaystyle (\theta )=p=\mu }$

${\displaystyle b''(\theta )={\frac {e^{\theta }}{1+e^{\theta }}}-\left({\frac {e^{\theta }}{1+e^{\theta }}}\right)^{2}}$

This give us

${\displaystyle V(\mu )=\mu (1-\mu )}$

Example – Poisson

Let ${\displaystyle y\sim {\text{Poisson}}(\lambda )}$, then we express the density of the Poisson distribution in exponential family form,

${\displaystyle f(y)=\exp(y\ln \lambda -\ln \lambda )}$

${\displaystyle \theta =\ln \lambda =}$ which gives us ${\displaystyle \lambda =e^{\theta }}$

${\displaystyle b(\theta )=e^{\theta }}$ and

${\displaystyle b'(\theta )=e^{\theta }=\lambda =\mu }$

${\displaystyle b''(\theta )=e^{\theta }=\mu }$

This give us

${\displaystyle V(\mu )=\mu }$

Here we see the central property of Poisson data, that the variance is equal to the mean.

Example – Gamma

The Gamma distribution and density function can be expressed under different parametrizations. We will use the form of the gamma with parameters ${\displaystyle (\mu ,\nu )}$

${\displaystyle f_{\mu ,\nu }(y)={\frac {1}{\Gamma (\nu )y}}\left({\frac {\nu y}{\mu }}\right)^{\nu }e^{\frac {\nu y}{\mu }}}$

Then in exponential family form we have

${\displaystyle f_{\mu ,\nu }(y)=\exp \left({\frac {-{\frac {1}{\mu }}y+\ln({\frac {1}{\mu }})}{\frac {1}{\nu }}}+\ln \left({\frac {\nu ^{\nu }y^{\nu -1}}{\Gamma (\nu )}}\right)\right)}$

${\displaystyle \theta ={\frac {-1}{\mu }}\rightarrow \mu ={\frac {-1}{\theta }}}$

${\displaystyle \phi ={\frac {1}{\nu }}}$

${\displaystyle b(\theta )=-ln(-\theta )}$

${\displaystyle b'(\theta )={\frac {-1}{\theta }}={\frac {-1}{\frac {-1}{\mu }}}=\mu }$

${\displaystyle b''(\theta )={\frac {1}{\theta ^{2}}}=\mu ^{2}}$

And we have ${\displaystyle V(\mu )=\mu ^{2}}$

Application – weighted least squares

A very important application of the variance function is its use in parameter estimation and inference when the response variable is of the required exponential family form as well as in some cases when it is not (which we will discuss in quasi-likelihood). Weighted least squares (WLS) is a special case of generalized least squares. Each term in the WLS criterion includes a weight that determines that the influence each observation has on the final parameter estimates. As in regular least squares, the goal is to estimate the unknown parameters in the regression function by finding values for parameter estimates that minimize the sum of the squared deviations between the observed responses and the functional portion of the model.

While WLS assumes independence of observations it does not assume equal variance and is therefore a solution for parameter estimation in the presence of heteroscedasticity. The Gauss–Markov theorem and Aitken demonstrate that the best linear unbiased estimator (BLUE), the unbiased estimator with minimum variance, has each weight equal to the reciprocal of the variance of the measurement.

In the GLM framework, our goal is to estimate parameters ${\displaystyle \beta }$, where ${\displaystyle Z=g(E[y\mid X])=X\beta }$. Therefore, we would like to minimize ${\displaystyle (Z-XB)^{T}W(Z-XB)}$ and if we define the weight matrix W as

${\displaystyle \underbrace {W} _{n\times n}={\begin{bmatrix}{\frac {1}{\phi V(\mu _{1})g'(\mu _{1})^{2}}}&0&\cdots &0&0\\0&{\frac {1}{\phi V(\mu _{2})g'(\mu _{2})^{2}}}&0&\cdots &0\\\vdots &\vdots &\vdots &\vdots &0\\\vdots &\vdots &\vdots &\vdots &0\\0&\cdots &\cdots &0&{\frac {1}{\phi V(\mu _{n})g'(\mu _{n})^{2}}}\end{bmatrix}},}$

where ${\displaystyle \phi ,V(\mu ),g(\mu )}$ are defined in the previous section, it allows for iteratively reweighted least squares (IRLS) estimation of the parameters. See the section on iteratively reweighted least squares for more derivation and information.

Also, important to note is that when the weight matrix is of the form described here, minimizing the expression ${\displaystyle (Z-XB)^{T}W(Z-XB)}$ also minimizes the Pearson distance. See Distance correlation for more.

The matrix W falls right out of the estimating equations for estimation of ${\displaystyle \beta }$. Maximum likelihood estimation for each parameter ${\displaystyle \beta _{r},1\leq r\leq p}$, requires

${\displaystyle \sum _{i=1}^{n}{\frac {\partial l_{i}}{\partial \beta _{r}}}=0}$, where ${\displaystyle \operatorname {l} (\theta ,y,\phi )=\log(\operatorname {f} (y,\theta ,\phi ))={\frac {y\theta -b(\theta )}{\phi }}-c(y,\phi )}$ is the log-likelihood.

Looking at a single observation we have,

${\displaystyle {\frac {\partial l}{\partial \beta _{r}}}={\frac {\partial l}{\partial \theta }}{\frac {\partial \theta }{\partial \mu }}{\frac {\partial \mu }{\partial \eta }}{\frac {\partial \eta }{\partial \beta _{r}}}}$

${\displaystyle {\frac {\partial \eta }{\partial \beta _{r}}}=x_{r}}$

${\displaystyle {\frac {\partial l}{\partial \theta }}={\frac {y-b'(\theta )}{\phi }}={\frac {y-\mu }{\phi }}}$

${\displaystyle {\frac {\partial \theta }{\partial \mu }}={\frac {\partial b'^{-1}(\mu )}{\mu }}={\frac {1}{b''(b'(\mu ))}}={\frac {1}{V(\mu )}}}$

This gives us

${\displaystyle {\frac {\partial l}{\partial \beta _{r}}}={\frac {y-\mu }{\phi V(\mu )}}{\frac {\partial \mu }{\partial \eta }}x_{r}}$, and noting that

${\displaystyle {\frac {\partial \eta }{\partial \mu }}=g'(\mu )}$ we have that

${\displaystyle {\frac {\partial l}{\partial \beta _{r}}}={\frac {y-\mu }{\phi V(\mu )}}W{\frac {\partial \eta }{\partial \mu }}x_{r}}$

The Hessian matrix is determined in a similar manner and can be shown to be,

${\displaystyle H=X^{T}(y-\mu )\left[{\frac {\partial }{\beta _{s}}}W{\frac {\partial }{\beta _{r}}}\right]-X^{T}WX}$

Noticing that the Fisher Information (FI),

${\displaystyle {\text{FI}}=-E[H]=X^{T}WX}$, allows for asymptotic approximation of ${\displaystyle {\hat {\beta }}}$

${\displaystyle {\hat {\beta }}\sim N_{p}(\beta ,(X^{T}WX)^{-1})}$, and hence inference can be performed.

Application – quasi-likelihood

Because most features of GLMs only depend on the first two moments of the distribution, rather than the entire distribution, the quasi-likelihood can be developed by just specifying a link function and a variance function. That is, we need to specify

– Link function: ${\displaystyle E[y]=\mu =g^{-1}(\eta )}$

– Variance function: ${\displaystyle V(\mu ){\text{, where the }}\operatorname {Var} _{\theta }(y)=\sigma ^{2}V(\mu )}$

With a specified variance function and link function we can develop, as alternatives to the log-likelihood function, the score function, and the Fisher information, a quasi-likelihood, a quasi-score, and the quasi-information. This allows for full inference of ${\displaystyle \beta }$.

Quasi-likelihood (QL)

Though called a quasi-likelihood, this is in fact a quasi-log-likelihood. The QL for one observation is

${\displaystyle Q_{i}(\mu _{i},y_{i})=\int _{y_{i}}^{\mu _{i}}{\frac {y_{i}-t}{\sigma ^{2}V(t)}}\,dt}$

And therefore the QL for all n observations is

${\displaystyle Q(\mu ,y)=\sum _{i=1}^{n}Q_{i}(\mu _{i},y_{i})=\sum _{i=1}^{n}\int _{y_{i}}^{\mu _{i}}{\frac {y-t}{\sigma ^{2}V(t)}}\,dt}$

From the QL we have the quasi-score

Quasi-score (QS)

Recall the score function, U, for data with log-likelihood ${\displaystyle \operatorname {l} (\mu \mid y)}$ is

${\displaystyle U={\frac {\partial l}{d\mu }}.}$

We obtain the quasi-score in an identical manner,

${\displaystyle U={\frac {y-\mu }{\sigma ^{2}V(\mu )}}}$

Noting that, for one observation the score is

${\displaystyle {\frac {\partial Q}{\partial \mu }}={\frac {y-\mu }{\sigma ^{2}V(\mu )}}}$

The first two Bartlett equations are satisfied for the quasi-score, namely

${\displaystyle E[U]=0}$

and

${\displaystyle \operatorname {Cov} (U)+E\left[{\frac {\partial U}{\partial \mu }}\right]=0.}$

In addition, the quasi-score is linear in y.

Ultimately the goal is to find information about the parameters of interest ${\displaystyle \beta }$. Both the QS and the QL are actually functions of ${\displaystyle \beta }$. Recall, ${\displaystyle \mu =g^{-1}(\eta )}$, and ${\displaystyle \eta =X\beta }$, therefore,

${\displaystyle \mu =g^{-1}(X\beta ).}$

Quasi-information (QI)

The quasi-information, is similar to the Fisher information,

${\displaystyle i_{b}=-\operatorname {E} \left[{\frac {\partial U}{\partial \beta }}\right]}$

QL,QS,QI as functions of ${\displaystyle \beta }$

The QL, QS and QI all provide the building blocks for inference about the parameters of interest and therefore it is important to express the QL, QS and QI all as functions of ${\displaystyle \beta }$.

Recalling again that ${\displaystyle \mu =g^{-1}(X\beta )}$, we derive the expressions for QL,QS and QI parametrized under ${\displaystyle \beta }$.

Quasi-likelihood in ${\displaystyle \beta }$,

${\displaystyle Q(\beta ,y)=\int _{y}^{\mu (\beta )}{\frac {y-t}{\sigma ^{2}V(t)}}\,dt}$

The QS as a function of ${\displaystyle \beta }$ is therefore

${\displaystyle U_{j}(\beta _{j})={\frac {\partial }{\partial \beta _{j}}}Q(\beta ,y)=\sum _{i=1}^{n}{\frac {\partial \mu _{i}}{\partial \beta _{j}}}{\frac {y_{i}-\mu _{i}(\beta _{j})}{\sigma ^{2}V(\mu _{i})}}}$

${\displaystyle U(\beta )={\begin{bmatrix}U_{1}(\beta )\\U_{2}(\beta )\\\vdots \\\vdots \\U_{p}(\beta )\end{bmatrix}}=D^{T}V^{-1}{\frac {(y-\mu )}{\sigma ^{2}}}}$

Where,

${\displaystyle \underbrace {D} _{n\times p}={\begin{bmatrix}{\frac {\partial \mu _{1}}{\partial \beta _{1}}}&\cdots &\cdots &{\frac {\partial \mu _{1}}{\partial \beta _{p}}}\\{\frac {\partial \mu _{2}}{\partial \beta _{1}}}&\cdots &\cdots &{\frac {\partial \mu _{2}}{\partial \beta _{p}}}\\\vdots \\\vdots \\{\frac {\partial \mu _{m}}{\partial \beta _{1}}}&\cdots &\cdots &{\frac {\partial \mu _{m}}{\partial \beta _{p}}}\end{bmatrix}}\underbrace {V} _{n\times n}=\operatorname {diag} (V(\mu _{1}),V(\mu _{2}),\ldots ,\ldots ,V(\mu _{n}))}$

The quasi-information matrix in ${\displaystyle \beta }$ is,

${\displaystyle i_{b}=-{\frac {\partial U}{\partial \beta }}=\operatorname {Cov} (U(\beta ))={\frac {D^{T}V^{-1}D}{\sigma ^{2}}}}$

Obtaining the score function and the information of ${\displaystyle \beta }$ allows for parameter estimation and inference in a similar manner as described in Application – weighted least squares.

Non-parametric regression analysis

A scattor plot of years in the major league against salary (x$1000). The line is the trend in the mean. The plot demonstrates that the variance is not constant.

The smoothed conditional variance against the smoothed conditional mean. The quadratic shape is indicative of the Gamma Distribution. The variance function of a Gamma is V(${\displaystyle \mu }$) = ${\displaystyle \mu ^{2}}$

Non-parametric estimation of the variance function and its importance, has been discussed widely in the literature^[5][6][7]In non-parametric regression analysis, the goal is to express the expected value of your response variable(y) as a function of your predictors (X). That is we are looking to estimate a mean function, ${\displaystyle g(x)=\operatorname {E} [y\mid X=x]}$ without assuming a parametric form. There are many forms of non-parametric smoothing methods to help estimate the function ${\displaystyle g(x)}$. An interesting approach is to also look at a non-parametric variance function, ${\displaystyle g_{v}(x)=\operatorname {Var} (Y\mid X=x)}$. A non-parametric variance function allows one to look at the mean function as it relates to the variance function and notice patterns in the data.

${\displaystyle g_{v}(x)=\operatorname {Var} (Y\mid X=x)=\operatorname {E} [y^{2}\mid X=x]-\left[\operatorname {E} [y\mid X=x]\right]^{2}}$

An example is detailed in the pictures to the right. The goal of the project was to determine (among other things) whether or not the predictor, number of years in the major leagues (baseball,) had an effect on the response, salary, a player made. An initial scatter plot of the data indicates that there is heteroscedasticity in the data as the variance is not constant at each level of the predictor. Because we can visually detect the non-constant variance, it useful now to plot ${\displaystyle g_{v}(x)=\operatorname {Var} (Y\mid X=x)=\operatorname {E} [y^{2}\mid X=x]-\left[\operatorname {E} [y\mid X=x]\right]^{2}}$, and look to see if the shape is indicative of any known distribution. One can estimate ${\displaystyle \operatorname {E} [y^{2}\mid X=x]}$ and ${\displaystyle \left[\operatorname {E} [y\mid X=x]\right]^{2}}$ using a general smoothing method. The plot of the non-parametric smoothed variance function can give the researcher an idea of the relationship between the variance and the mean. The picture to the right indicates a quadratic relationship between the mean and the variance. As we saw above, the Gamma variance function is quadratic in the mean.

Notes

1. Muller and Zhao (1995). "On a semi parametric variance function model and a test for heteroscedasticity". The Annals of Statistics. 23 (3): 946–967. doi:10.1214/aos/1176324630. JSTOR 2242430.
2. Muller, Stadtmuller and Yao (2006). "Functional Variance Processes". Journal of the American Statistical Association. 101 (475): 1007–1018. doi:10.1198/016214506000000186. JSTOR 27590778. S2CID 13712496.
3. Wedderburn, R.W.M. (1974). "Quasi-likelihood functions, generalized linear models, and the Gauss–Newton Method". Biometrika. 61 (3): 439–447. doi:10.1093/biomet/61.3.439. JSTOR 2334725.
4. McCullagh, Peter; Nelder, John (1989). Generalized Linear Models (second ed.). London: Chapman and Hall. ISBN 0-412-31760-5.
5. Muller and StadtMuller (1987). "Estimation of Heteroscedasticity in Regression Analysis". The Annals of Statistics. 15 (2): 610–625. doi:10.1214/aos/1176350364. JSTOR 2241329.
6. Cai and Wang, T.; Wang, Lie (2008). "Adaptive Variance Function Estimation in Heteroscedastic Nonparametric Regression". The Annals of Statistics. 36 (5): 2025–2054. arXiv:0810.4780. Bibcode:2008arXiv0810.4780C. doi:10.1214/07-AOS509. JSTOR 2546470. S2CID 9184727.
7. Rice and Silverman (1991). "Estimating the Mean and Covariance structure nonparametrically when the data are curves". Journal of the Royal Statistical Society. 53 (1): 233–243. JSTOR 2345738.

References

• McCullagh, Peter; Nelder, John (1989). Generalized Linear Models (second ed.). London: Chapman and Hall. ISBN 0-412-31760-5.
• Henrik Madsen and Poul Thyregod (2011). Introduction to General and Generalized Linear Models. Chapman & Hall/CRC. ISBN 978-1-4200-9155-7.

External links

• Media related to Variance function at Wikimedia Commons