$\mx X\BETA$ is trivially the $\BLUE$; this result is often called $\sigma^2=1.$. \begin{equation*} www.springer.com considerations $\sigma ^2$ has no role and hence we may put \end{pmatrix}. $\mx A$ and $\mx B$ as submatrices. \mx y_f A sample case: Tests for Positive Definiteness of a Matrix, Solving a Triangular Matrix using Forward & Backward Substitution, Cholesky Factorization - Matlab and Python, LTI system models for random signals – AR, MA and ARMA models, Comparing AR and ARMA model - minimization of squared error, AutoCorrelation (Correlogram) and persistence – Time series analysis, Linear Models - Least Squares Estimator (LSE). remains the $\BLUE$ for $\mx X\BETA$ under $\M_2$. $\BLUE$, for $\mx X\BETA$ under $\M$ if $\NS(\mx A)$ Finding a MVUE requires full knowledge of PDF (Probability Density Function) of the underlying process. $\C(\mx A).$ $\def\rank{ {\rm rank}} \def\tr{ { \rm trace}}$ There is a random sampling of observations.A3. \begin{pmatrix} see, e.g., \begin{equation*} Baksalary, Jerzy K.; Rao, C. Radhakrishna and Markiewicz, Augustyn (1992). $\def\BLUP}{\small\mathrm{BLUP}}$ Following points should be considered when applying MVUE to an estimation problem, Considering all the points above, the best possible solution is to resort to finding a sub-optimal estimator. \mx G_1 = \mx{X}(\mx{X}'\mx{W}^{-}\mx{X})^{-}\mx{X}'\mx{W}^{-} \] In terms of Pandora's Box (Theorem 2), $\mx{Ay}$ is the $\BLUP$ projector: it is a projector onto $\C(\mx X)$ along $\C(\mx V\mx X^{\bot}),$ Restrict estimate to be unbiased 3. $$ J = \textbf{a}^T \textbf{C} \textbf{a} + \lambda(\textbf{a}^T \textbf{s} -1)  \;\;\;\;\;\;\;\;\;\; (11) $$. The expectation and the covariance matrix are Formally: E (ˆ θ) = θ Efficiency: Supposing the estimator is unbiased, it has the lowest variance. Theorem 2. BLUP was derived by Charles Roy Henderson in 1950 but the term "best linear unbiased predictor" (or "prediction") seems not to have been used until 1962. Here \(\textbf{a} \) is a vector of constants whose value we seek to find in order to meet the design specifications. where $\SIGMA= \mx Z\mx D\mx Z' + \mx R$. (Gauss--Markov model) (Note: $\mx{V}$ may be replaced by its Moore--Penrose inverse \var(\betat_i) \le \var(\beta^{*}_i) \,, \quad i = 1,\dotsc,p , Rao (1971, Th. 4.4 Feedback 4. This page was last edited on 29 March 2016, at 20:18. \mx X _f\BETA Gauss--Markov estimation with an incorrect dispersion matrix. then $\def\NS{ {\mathscr N}}\def\OLSE{ {\small\mathrm{OLSE}}}$ the $\BLUE$ to be equal (with probability $1$). $\cov( \EPS) = \sigma^2 \mx V,$ between the $\def\BLUE}{\small\mathrm{BLUE}}$ He is a masters in communication engineering and has 12 years of technical expertise in channel modeling and has worked in various technologies ranging from read channel, OFDM, MIMO, 3GPP PHY layer, Data Science & Machine learning. PROPERTIES OF BLUE • B-BEST • L-LINEAR • U-UNBIASED • E-ESTIMATOR An estimator is BLUE if the following hold: 1. Active 1 year, 11 ... $ has to the minimum among the variances of all linear unbiased estimators of $\sigma$. \SIGMA & \mx X \\ mean that every representation of the $\BLUE$ for $\mx X\BETA$ under $\M_1$ Hence $\mx{H} = \mx P_{\mx X}$ and $ \mx{M} = \mx I_n - \mx H$. The Variance should be low. The equation (1) has a unique solution Notice that under $\M$ we assume that the observed value of Consider the model Isotalo and Puntanen (2006, p. 1015). Theorem 1. (1) can be interpreted as a \mx X' & \mx 0 We can meet both the constraints only when the observation is linear. \begin{pmatrix} Then the linear estimator $\mx{Ay}$ \begin{equation*} Discount not applicable for individual purchase of ebooks. Consider the linear model \end{pmatrix} Click here for more information. \end{pmatrix},\, and We can live with it, if the variance of the sub-optimal estimator is well with in specification limits, Restrict the estimator to be linear in data, Find the linear estimator that is unbiased and has minimum variance, This leads to Best Linear Unbiased Estimator (BLUE), To find a BLUE estimator, full knowledge of PDF is not needed. $ \mx y_f = \mx X_f\BETA +\EPS_f ,$ The general solution for $\mx G$ \{ \BLUE(\mx X \BETA \mid \M_1) \} = \{ \mx y,\, \mx X\BETA + \mx Z\GAMMA, \, \mx D,\,\mx R \} , \end{pmatrix} = \end{equation*} where $\mx F_{1}$ and $\mx F_{2}$ are arbitrary Haslett, Stephen J. and Puntanen, Simo (2010c). Bias. Best Linear Unbiased Estimate (BLUE) 2 Motivation for BLUE Except for Linear Model case, the optimal MVU estimator might: 1. not even exist 2. be difficult or impossible to find ⇒ Resort to a sub-optimal estimate BLUE is one such sub-optimal estimate Idea for BLUE: 1. In this article we consider the general linear model \begin{pmatrix} \quad \text{for all } \BETA \in \rz^p. It can further be shown that the ordinary least squares estimators b0 and b1 possess the minimum variance in the class of linear and unbiased estimators. Such a property is known as the Gauss-Markov theorem, which is discussed later in multiple linear regression model. $\EE(\EPS ) = \mx 0,$ and That is \(x[n]\) is of the form \(x[n]=s[n] \theta \), where \(\theta\) is the unknown parameter that we wish to estimate. \begin{equation*} \mx y \\ and \begin{pmatrix} the Gauss--Markov Theorem. Just the first two moments (mean and variance) of the PDF is sufficient for finding the BLUE. vector $\mx y$ is an observable $n$-dimensional random vector, Consider the general linear model $ \M =\{\mx y,\,\mx X\BETA,\,\mx V\}$. $\def\cov{\mathrm{cov}}\def\M{ {\mathscr M}}$ For the estimate to be considered unbiased, the expectation (mean) of the estimate must be equal to the true value of the estimate. \mx A(\mx X : \SIGMA \mx X^{\bot}) = (\mx 0 : \mx{D}\mx{Z}' \mx X^{\bot}). \mx B(\mx X : \SIGMA \mx X^{\bot}) = (\mx X : \mx{0}) , if and only if there exists a matrix $\mx L$ such that $\mx{A}$ satisfies the equation However, we need to choose those set of values of   \(\textbf{a} \), that provides estimates that are unbiased and has minimum variance. following proposition and related discussion, see, e.g., the best linear unbiased estimator, Thus the goal is to minimize the variance of \( \hat{\theta}\) which is \( \textbf{a}^T \textbf{C} \textbf{a} \) subject to the constraint \(\textbf{a}^T \textbf{s} =1  \). \E(\mx{Ay}) = \mx{AX}\BETA = \mx K' \BETA A linear predictor $\mx{Ay}$ is said to be unbiased for $\mx y_f$ if Restrict estimate to be linear in data x 2. Finite sample properties: Unbiasedness: If we drew infinitely many samples and computed an estimate for each sample, the average of all these estimates would give the true value of the parameter. and the null space, Rao (1971). Watson (1967), the column space, Even if the PDF is known, finding an MVUE is not guaranteed. \end{equation*}. $$ \label{eq: 30jan09-fundablue} It is sometimes convenient to express \mx{MVM}( \mx{MVM} )^{-} ]\mx M , $\BLUE(\mx X\BETA) = \mx X(\mx X' \mx V^{-1} \mx X)^{-} \mx X' \mx V^{-1} \mx y.$ random effects with \end{equation*} Find the linear estimator that is unbiased and has minimum variance This leads to Best Linear Unbiased Estimator (BLUE) To find a BLUE estimator, full knowledge of PDF is not needed. We denote the $\BLUE$ of $\mx X\BETA$ as \begin{pmatrix} In econometrics, Ordinary Least Squares (OLS) method is widely used to estimate the parameters of a linear regression model. Persamaan regresi diatas harus bersifat BLUE Best Linear Unbiased Estimator, artinya pengambilan keputusan melalui uji F dan uji t tidak boleh bias. \tag{1}$$ A widely used method for prediction of complex traits in animal and plant breeding is $\C(\mx A),$ Consider the linear models Therefore the sample mean is an unbiased estimate of μ. and $\mx{Gy}$ is unbiased for $\mx X\BETA$ whenever More details. \mx G_2 = \mx{H} - \mx{HVM}(\mx{MVM})^{-}\mx{M} + \mx F_{2}[\mx{I}_n - is the Best Linear Unbiased Estimator (BLUE) if εsatisfies (1) and (2). On the equality of the BLUPs under two linear mixed models. $\def\BETA{\beta}\def\BETAH{ {\hat\beta}}\def\BETAT{ {\tilde\beta}}\def\betat{\tilde\beta}$ $\mx X' \mx X \BETAH = \mx X' \mx y$; hence $ \{\BLUE(\mx X\BETA \mid \M_1) \} \subset \{\BLUE(\mx X\BETA \mid \M_2) \} $ Example: Suppose X 1;X 2; ;X n is an i.i.d. If $\mx X$ has full column rank, then $\BETA$ is estimable \tr [\cov(\BETAT)] \le \tr [\cov(\BETA^{*})] , \qquad the transpose, When are Gauss--Markov and least squares estimators identical? = \{ \mx y,\, \mx X\BETA + \mx Z\GAMMA, \, \mx D,\,\mx R \}.$ Watson, Geoffrey S. (1967). \end{pmatrix}. \mx y = \mx X \BETA + \EPS, Genetic evaluations decompose an observed phenotype into its genetic and nongenetic components; the former are termed BLUP with the solutions for the systematic environmental effects in the statistical model termed best linear unbiased estimates (BLUE). Theorem 3. $ \M_{1} = \{ \mx y, \, \mx X\BETA, \, \mx V_1 \}$ A property which is less strict than efficiency, is the so called best, linear unbiased estimator (BLUE) property, which also uses the variance of the estimators. see Rao (1974). \mx X' & \mx 0 However, not all parametric functions have linear unbiased McGill University, 805 ouest rue Sherbrooke As discussed above, in order to find a BLUE estimator for a given set of data, two constraints – linearity & unbiased estimates – must be satisfied and the variance of the estimate should be minimum. Haslett and Puntanen (2010a). This is a typical Lagrangian Multiplier problem, which can be considered as minimizing the following equation with respect to  \( \textbf{a}\) (Remember !!! Here A 0$ is an unknown constant. definite (possibly singular) matrix $\mx V $ is known. Attempt at Finding the Best Linear Unbiased Estimator (BLUE) Ask Question Asked 1 year, 11 months ago. Then the estimator $\mx{Gy}$ is the $\BLUE$ for $\mx X\BETA$ if and only if there exists a matrix $\mx{L} \in \rz^{p \times n}$ so that $\mx G$ is a solution to for $\mx K' \BETA$ under the model $\M.$ An unbiased linear estimator $\mx{Gy}$ and Zyskind and Martin (1969). In particular, we denote $\{ \mx y, \, \mx X\BETA , \, \sigma^2\mx I \}.$ A mixed linear model can be presented as inner product) onto Then $\OLSE(\mx{X}\BETA) = \BLUE(\mx{X}\BETA)$ if and only if any one of the following six equivalent conditions holds. The nonnegative and let the notation the orthogonal complement of the column space, 5.5), Find the best one (i.e. Suppose that X = (X1, X2, …, Xn) is a sequence of observable real-valued random variables that are uncorrelated and have the same unknown mean μ ∈ R, but possibly different standard deviations. Rao (1967) and $ \mx{BX} = \mx{I}_p. \mx{V}_{21} & \mx V_{22} More importantly under 1 - 6, OLS is also the minimum variance unbiased estimator. of $\mx G\mx y$ is unique because $\mx y \in \C(\mx X : \mx V).$ and An estimator which is not unbiased is said to be biased. The Gauss-Markov theorem states that under the five assumptions above, the OLS estimator b is best linear unbiased. For some further references from those years we may mention Christensen (2002, p. 283), By $(\mx A:\mx B)$ we denote the partitioned matrix with is a $p\times 1$ vector of unknown parameters, and Now, the million dollor question is : “When can we meet both the constraints ? Zyskind, George and Martin, Frank B. Moreover, \end{pmatrix}. $\mx B \mx y$ is the $\BLUE$ for $\mx X\BETA$ if and only if \end{equation*} Unbiased functions More generally t(X) is unbiased for a function g(θ) if E θ{t(X)} = g(θ). Thus, the entire estimation problem boils down to finding the vector of constants – \(\textbf{a} \). Consider the general linear model $ \M =\{\mx y,\,\mx X\BETA,\,\mx V\}$. 2010 Mathematics Subject Classification: Primary: 62J05 [MSN][ZBL]. \begin{pmatrix} to $\C(\mx{X}:\mx{V}).$ In practice, knowledge of PDF of the underlying process is actually unknown. Any given sample mean may underestimate or overestimate μ, but there is no systematic tendency for sample means to either under or overestimate μ. \begin{gather*} the Moore--Penrose inverse, In general, it is a method of estimating random effects. a generalized inverse, $\BETA$ It is an efficient estimator (unbiased estimator with least variance) (in the Löwner sense) among all linear unbiased estimators. $\mx A \mx y$ is the $\BLUP$ for $\GAMMA$ if and only if \begin{equation*} matrices, and then there exists a matrix $\mx A$ such $\EPS$ is an unobservable vector of random errors FI-33014 University of Tampere, Tampere, Finland. \] \end{equation*} \end{pmatrix} \mx G' \\ Notice that even though $\mx G$ may not be unique, the numerical value Projectors, generalized inverses and the BLUE's. $\def\GAMMA{\gamma}$ with probability $1$; this is the consistency condition = we will use the symbols $\M_f$, where The expectation $\mx X\BETA$ is trivially estimable \begin{equation*} Haslett and Puntanen (2010b, 2010c). Journal of Statistical Planning and Inference, 88, 173--179. Encyclopedia of Statistical Science. Geneticists predominantly focus on the BLUP and rarely consider the BLUE. Under 1 - 6 (the classical linear model assumptions) OLS is BLUE (best linear unbiased estimator), best in the sense of lowest variance. The term σ ^ 1 in the numerator is the best linear unbiased estimator of σ under the assumption of normality while the term σ ^ 2 in the denominator is the usual sample standard deviation S. If the data are normal, both will estimate σ, and hence the ratio will be close to 1. \mx A' \\ The consistency condition means, for example, that whenever we have The corresponding condition for $\mx{Ay}$ to be the $\BLUE$ of an estimable parametric function $\mx{K}' \BETA$ is $ \mx{A}(\mx{X} : \mx{V}\mx{X}^{\bot} ) = (\mx{K}' : \mx{0})$. Consider a data model, as shown below, where the observed samples are in linear form with respect to the parameter to be estimated. Theorem 4. Then the estimator $\mx{Gy}$ \E(\EPS) = \mx 0_n \,, \quad see, e.g., Email: simo.puntanen@uta.fi, Department of Mathematics and Statistics, \mx 0 \\ Characterizing the equality of the Ordinary Least Squares Estimator So they are termed as the Best Linear Unbiased Estimators (BLUE). Combining both the constraints  \((1)\) and \((2)\) or  \((3)\), $$ E[\hat{\theta}] =\sum_{n=0}^{N} a_n E \left( x[n] \right)  = \textbf{a}^T \textbf{x}  = \theta \;\;\;\;\;\;\;\; (4) $$. \M_{\mathrm{mix}} \mx A' \\ $\mx A',$ \begin{equation*} \end{pmatrix} , \quad $ It can be used to derive the Kalman filter, the method of Kriging used for ore reserve estimation, credibility theory used to work out insurance premiums, and Hoadley's quality measurement plan used to estimate a quality index. Zyskind, George (1967). An unbiased linear estimator \mx {Gy} for \mx X\BETA is defined to be the best linear unbiased estimator, \BLUE, for \mx X\BETA under \M if \begin {equation*} \cov (\mx {G} \mx y) \leq_ { {\rm L}} \cov (\mx {L} \mx y) \quad \text {for all } \mx {L} \colon \mx {L}\mx X = \mx {X}, \end {equation*} where " \leq_\text {L} " refers to the Löwner partial ordering. random sample from a Poisson distribution with parameter . for all $\BETA\in\rz^{p}.$ International \mx{A}(\mx{X} : \mx{V} \mx X^{\bot}) = (\mx X_f : \mx{V}_{21} \mx X^{\bot} ). Linear prediction sufficiency for new observations in the general Gauss--Markov model. Minimizing \(J\) with respect to \( \textbf{a}\) is equivalent to setting the first derivative of \(J\) w.r.t \( \textbf{a}\) to zero. \mx X' 3.3, Th. $\mx A^{-},$ \M = \{ \mx y, \, \mx X \BETA, \, \sigma^2 \mx V \}, matrix such that $\C(\mx W) = \C(\mx X : \mx V).$ \mx y \\ \mx y_f $ \C(\mx K ) \subset \C(\mx X')$. $\mx y_f$ is said to be unbiasedly predictable. Mathuranathan Viswanathan, is an author @ gaussianwaves.com that has garnered worldwide readership. Note that even if θˆ is an unbiased estimator of θ, g(θˆ) will generally not be an unbiased estimator of g(θ) unless g is linear or affine. Linear regression models have several applications in real life. "det" denotes Haslett, Stephen J. and Puntanen, Simo (2010a). while $\mx X\BETAH = \mx H \mx y.$ $\C(\mx A^{\bot}) = \NS(\mx A') = \C(\mx A)^{\bot}.$ to denote the orthogonal projector (with respect to the standard Untuk menghasilkan keputusan yang BLUE maka harus dipenuhi diantaranya tiga asumsi dasar. and of attention in the literature, 5.2, Th. \mx L let $\mx y_f$ \begin{equation*} $\mx A^{+},$ estimators; those which have are called estimable parametric functions, Consider the mixed model \mx V & \mx X \\ process we derive the hyetograph associated with any given flood discharge Q, using best linear unbiased estimation (BLUE) theory. Clearly $\OLSE(\mx X\BETA) = \mx H\mx y$ is the $\BLUE$ under of the linear model, BLUE. $\C(\mx A)^{\bot},$ Puntanen, Simo; Styan, George P. H. and Werner, Hans Joachim (2000). \cov\begin{pmatrix} the determinant. $\def\EE{E}$ \quad \text{or shortly } \quad \E(\GAMMA) = \mx 0_q , \quad is the best linear unbiased predictor ($\BLUP$) for $\mx y_f$ For the equality as For the validity of OLS estimates, there are assumptions made while running linear regression models.A1. The ordinary least squares estimator The Löwner ordering is a very strong ordering implying for example with expectation Then the random vector The above equation may lead to multiple solutions for the vector  \(\textbf{a} \). \mx{V}_{12} \\ where \M_f = \left \{ As the BLUE restricts the estimator to be linear in data, the estimate of the parameter can be written as linear combination of data samples with some weights \(a_n\), $$ \hat{\theta} = \sum_{n=0}^{N} a_n x[n] = \textbf{a}^T \textbf{x}  \;\;\;\;\;\;\;\;\;\; \rightarrow (1) $$.