Skip to main content

Statistical Inference I ( Theory of estimation : Efficiency)

🔖Statistical Inference I ( Theory of estimation : Efficiency) 

In this article we see the  terms: 

I. Efficiency.

II. Mean Square Error.

III. Consistency.


📚Efficiency: 

We know that  two unbiased estimator of parameter gives rise to infinitely many unbiased estimators of parameter. there if one of parameter have two estimators then the problem is to choose one of the best estimator among the class of unbiased estimators. in that case we need to some other criteria to to find out best estimator. therefore, that situation  we check the variability of that estimator, the measure of variability of estimator T around it mean is Var(T). hence If T is an Unbiased estimator of parameter then it's variance gives good precision. the variance is smaller then it give's greater precision.


📑 i. Efficient estimator: An estimator T is said to be an Efficient Estimator of 𝚹, if T is unbiased estimator of  𝛉. and it's variance is less than any other estimator. i.e. Var(T) < Var(T*).

where T* is any other estimator of  𝛉.

🔖ii. Relative Efficiency: If T1 and T2 are two unbiased estimators of parameter  𝛉 and E(T₁²) < ∞ and  E(T₂²) < ∞ then relative efficiency of T1  with respect to  T2 is denoted by Efficiency( T1 ,T2 ).

and T1  )/ V(T2 )

Remark: i. if relative efficiency =1 it means that  T1 and T2 are equal efficient estimators.

ii. If Relative Efficiency ( T1 and T2 ) < 1, it means that V( T1 ) <  V( T2 ). 

i.e. T1 is more efficient than T2 .

iii. If Relative Efficiency ( T1  and T2 ) > 1, it means that V( T1 ) >  V( T2 ). 

i.e. T2 is more efficient than  T1 

Example: 1. if X1 and X2 are two independent observation from normal distribution with mean 𝚹 and variance  𝛔² , then show thatT1 = ( X1 + X2)/2 , T2 = ( X1 + 2X2 )/3 are unbiased estimators of 𝚹 and also find the relative Efficiency (  T1 ,T2 ).

Solution: if  X1  and X2 are two independent observation from normal distribution with mean 𝚹 and variance  𝛔² 

E( X1 ) = E( X2 ) = 𝚹.

we have T1 = (X1 + X2)/2 , T2 = (X1 + 2X2 )/3

E(T1 ) = E[(X1 + X2 )/2] , E(T2 )= E [(X1 + 2X2 )/3].  we have E(X1 ) = E(X2) = 𝚹.

E(T1 ) = E[(𝚹+𝚹)/2] , E(T2 )= E [(𝚹 + 2𝚹 )/3

E(T1 ) = 2𝚹/2 , E(T2 )=  3𝚹 /3

E(T1 ) = 𝚹 , E(T2 ) =  𝚹 

Thus  T1 &T2 are unbiased estimators of parameter 𝚹.

now we obtain Variation of  T1 &T2 

V(T1 ) =V[ (X1 + X2 )/2] , V(T2)= (X1 + 2X2 )/3

V(T1 ) =[V(X1 )+V( X2 )] /4 , V(T2 )=[ V(X1 )+ 4V(X2 )] /3

V(T1 ) =[V(X1 )+V( X2 )] /4 , V(T2)=[ V(X1 )+ 4V(X2 )] /9

V(T1 ) =[𝛔² +𝛔² ] /4 , V(T2)=[ 𝛔² + 4𝛔² ] /9

V(T1 ) =2𝛔² /4 , V(T2)=5𝛔² /9

V(T1 ) =𝛔² /2 , V(T2)=5𝛔² /9

relative Efficiency  T1 relative to T2 is 

E (  T1 ,T2) =  V(  T1 )/V(T2)

                    = (𝛔² /2 ) /(5𝛔² /9)

                    = 9/10

                    = 0.9 < 1

therefore  E (  T1 ,T2) < 1 , means V( T1 ) =𝛔² /2  < V(T2)=5𝛔² /9 

Hence T1 is more efficient than T2 

 

Example 2. if X1, X2, ........Xn. random sample form normal distribution with mean 𝚹 and variance  𝛔² ,    T1 = sample mean i.e. x̄, and T2= sample median. find the  Efficiency of  T1 relative to T2 .   given that variance of median is  = V( T2)= (𝝅 𝛔² )/2n.    (for large n)

solution:  X1, X2, ........Xn. random sample form normal distribution with mean 𝚹 and variance  𝛔² then we have V(x̄) = V( T1 ) = 𝛔² /n. and  V( T2 )= (𝝅 𝛔² )/2n.

therefore the  Efficiency of T1 relative to T2  = E (  T1 ,T2 ) =  V( T1 )/V(T2 )

E ( T1 ,T2 ) =  V( T1 )/V(T2) 

                       =[ 𝛔² /n] / [ (𝝅 𝛔² )/2n]

                        = [𝛔² /n] x[2n/((𝝅 𝛔² )]

                        = (2/𝝅 )

                        =  0.6369 < 1 

                  therefore  E (   T1 ,T2 ) < 1 , means V( T1  ) =𝛔² /n <  V( T2  )= 𝝅𝛔² /2n

Hence  T1  is more efficient than  T2  

i.e. sample mean is more efficient than sample median.


III. Mean Square Error (M. S. E.)

let   T1  &  T2   be the  two estimators of parameter 𝚹, where  T1   is a unbiased estimator of 𝚹 &   T2  is a biased estimator of 𝚹. suppose the possible value of    T1  is spread   around  𝚹 and the value of  T2  is near to the parameter.  therefore in this case  T2   may be preferred than that of  T1  .

that case we need to study the variability of estimator around the parameter. for this we have to find out the variance of the estimator, which measure the variability of estimator around it's mean. if  T1  is an unbiased estimator of it's variance gives a good measure of precision. but if   T2   is biased estimator of parameter then variability of  T around 𝚹. as measure of it's precision and it is called as Mean Square Error (M. S. E.).

 Mean Square Error : an estimator T is a said to be a good estimator of 𝚹 if it's mean square error is minimum.

 i.e. E(T-𝚹)² ≤ E(T*-𝚹)²

where 𝚹. = T and T* is any other estimator .

this criteria known as mean square error.


Result: show that M. S. E. of T = MSE(T) = Var(T) + b² (T,𝚹) where b(.) is biased estimator.

Proof: 

 by definition of M. S. E. (T) = E(T-𝚹.)²

                                                = E[T-E(T)+E(T)-𝚹]²

   = E{T-E(T)}²+E{E(T)-𝚹}² + E{[T-E(T)]x[E(T)-𝚹]}

                                          = E{T-E(T)}²+E{E(T)-𝚹}² 

                                                 = Var(T) + b² (T,𝚹)

{Var(T)  = E{T-E(T)}²,   b² (T,𝚹) =E{E(T)-𝚹}² }

therefore  M. S. E. of T = MSE(T) = Var(T) + b² (T,𝚹).


Definition: Mean Square Error (M. S. E.) :

The M.S.E.  of estimator T of parameter 𝚹  is defined as 

    M. S. E. of T = MSE(T) = Var(T) + b² (T,𝚹)     if  T is biased estimator of  𝚹.

    M. S. E. of T = MSE(T) = Var(T)            

          if an T unbiased estimator of 𝚹.

this is the definition of M.S.E.


Example: 1. if   T1 & T2 be the two unbiased estimators of parameter 𝚹 with variance 𝞂1² and 𝞂2² respectively and having correlation ρ then find the best liner unbiased combination of T1 & T2  and also find the expression of variance of such combination.

Solution: we have two estimators  T1 & T2 are unbiased estimators of parameter 𝚹 with variance 𝞂1² and  𝞂2².

 E(T1 ) = 𝚹 , E(T2 ) =  𝚹 

V(T1 ) =  𝞂1² , V(T2)=  𝞂2².

and correlation is ρ

consider a linear combination of estimators   T1 & T2 is T = ɑ T1 + (1-ɑ) T2

now taking expectation E(T)
E(T)  = E[ɑ T1 + (1-ɑ) T2]
           =ɑ E[T1 ]+ (1-ɑ)E[ T2]
            = ɑ 𝚹 + (1-ɑ) 𝚹 
                =  ɑ 𝚹 + 1𝚹-ɑ𝚹 
            = 𝚹
E(T) = 𝚹
therefore T is an unbiased estimator of 𝚹
now we find variance of T
Var(T) =  V[ɑ T1 + (1-ɑ) T2]
   =ɑ²V( T1 ) + (1-ɑ)² V(T2) +2ɑ(1-ɑ)COV( T1, T2 )
  V(T)= ɑ² 𝞂1²+ (1-ɑ)²  𝞂2² +2ρ 𝞂1𝞂2 ɑ(1-ɑ)
           we find the value of ɑ and 1-ɑ the variance of (T)is minimise.
for that we find [dV(T) / da] = 0 and  [d²V(T) /da²] > 0.
now,  [dV(T) / da] = 0  i.e. differentiate w.r.t. a

 dV(T) / da  = {d/da} [  ɑ² 𝞂1²+ (1-ɑ)²  𝞂2² +2ɑ(1-ɑ)ρ 𝞂1 𝞂2]
=> 2ɑ 𝞂1² -2 (1-ɑ)  𝞂2² +2(1-2ɑ)ρ 𝞂1 𝞂2  = 0
              ɑ 𝞂1² - (1-ɑ)  𝞂2² + (1-2ɑ)ρ 𝞂1 𝞂2 = 0 
      ɑ 𝞂1² -  𝞂2² +ɑ 𝞂2² + ρ 𝞂1 𝞂2 -2ɑ ρ 𝞂1 𝞂2  = 0
          ɑ( 𝞂1²+𝞂2²-2  ρ 𝞂1 𝞂2 ) -  𝞂2² + ρ 𝞂1 𝞂2 = 0
                         ɑ( 𝞂1²+𝞂2²-2  ρ 𝞂1 𝞂2 ) =  𝞂2² - ρ 𝞂1 𝞂2 
    ɑ =  ( 𝞂2² - ρ 𝞂1 𝞂2 ) / ( 𝞂1²+𝞂2²-2ρ 𝞂1 𝞂2 )       
therefore    
 1-  ɑ= ( 𝞂1² - ρ 𝞂1 𝞂2 )/ ( 𝞂1²+𝞂2²-2 ρ 𝞂1 𝞂2 ) 
and 
d²V(T) /da² = 
{d/da} [ 2ɑ 𝞂1² -2 (1-ɑ)  𝞂2² +2(1-2ɑ)ρ 𝞂1 𝞂2 ]
                    = 2𝞂1²+2𝞂2²-4  ρ 𝞂1 𝞂2
                      =    2( 𝞂1²+𝞂2²-2 ρ 𝞂1 𝞂2 )
                    =2[V( T1 ) +V(T2) -2COV( T1, T2 )]
                    = 2V( T1 - T2 ) > 0
hence variance of (T) is minimum at the value of ɑ =  ( 𝞂2² - ρ 𝞂1 𝞂2 ) / ( 𝞂1²+𝞂2²-2ρ 𝞂1 𝞂2 )       
  1-  ɑ=( 𝞂1² - ρ 𝞂1 𝞂2 )/( 𝞂1²+𝞂2²-2 ρ 𝞂1 𝞂2 )  
for variance of T we put the values of  ɑ =  ( 𝞂2² - ρ 𝞂1 𝞂2 ) / ( 𝞂1²+𝞂2²-2ρ 𝞂1 𝞂2 )       
     1-  ɑ =  ( 𝞂1² - ρ 𝞂1 𝞂2 ) / ( 𝞂1²+𝞂2²-2 ρ 𝞂1 𝞂2 ).
now find the variance of T 
Var(T) =  V[ɑ T1 + (1-ɑ) T2]
   =ɑ²V( T1 ) + (1-ɑ)² V(T2) +2ɑ(1-ɑ)COV( T1, T2 )
 V(T)   = ɑ² 𝞂1²+ (1-ɑ)²  𝞂2² +2ɑ(1-ɑ) ρ 𝞂1 𝞂2.





Note that: Smaller M.S.E. i.e. variance is small means greater precision. hence while comparing two estimators T1 & T2 of  𝞗  we choose an estimator with smaller M.S.E. this used to modifies the formula of efficiency

Efficiency of T1 & T2 = M.S.E(T1 ) / M.S.E(T2 )

If efficiency (T1 , T2 ) < 1
then T1  is more efficient estimator than  T2 
i.e.  M.S.E(T1 )  <  M.S.E(T2 )

Locally Minimum Variance Unbiased Estimator (MVUE).
Definition: Let 𝞗0 ∈ 𝚯 and U(𝞗0) be the class of all unbiased estimates of θ such that     E(T²) <∞. AndT0∈ U(𝞗0) then T0 is called locally minimum variance unbiased estimator(LMVUE).
OR simply minimum variance unbiased estimator( SMVUE). IF E(T0-θ0)² ≤E(T -θ0)² holds for all T.

Uniformly Minimum Variance Unbiased Estimator (UMVUE)
Definition: Let U be the set of all unbiased estimates of T     of 𝞗 ∈ 𝚯, such that  E(T²) <∞. and estimate T0 ∈ U is called uniformly minimum variance unbiased estimator of 𝞗 if 
E(T0-θ)² ≤E(T -θ)² for all 𝞗 ∈ 𝚯 and every T Ɛ U.

Now we see the Definition of Minimum Variance Unbiased Estimator (MVUE)
Definition: if a statistics T=T(X1, X2, .....Xn) based on a Random sample of size n such that i) T is unbiased estimator for Ɣ(θ) for all 𝞗 ∈ 𝚯  ii) it has smallest variance among the class of all unbiased estimators of  Ɣ(θ), then T  is called  Minimum Variance Unbiased Estimator (MVUE)
OR
T is MVUE of  Ɣ(θ) if 
i)E(T) = Ɣ(θ)  for all 𝞗 ∈ 𝚯
ii) V(T) ≤ V(T*)
where T* is any other unbiased estimator of   Ɣ(θ).


Result: If UMVUE exist then it is unique.
Proof : Suppose that  T1 & T2 are two uniformly minimum variance unbiased estimator of parameter θ then 
we have     E(T1) =θ,   E(T2) =θ   for all θ ∈ 𝚯
and V(T1) =  V(T2)    for all θ ∈ 𝚯         ............1
now we define new estimator T =(T1 + T2 ) /2  which is unbiased estimator 
since E(T ) = E[(T1 + T2 ) /2]
                    =  (θ+θ)/2
                     =2θ /2
                    =θ
And variance of T 
Var(T) = Var[(T1 + T2 ) /2]
            = {V(T1 ) +V( T2 ) + 2 Cov( T1 , T2)}/4
            = {2V(T1 ) + 2 ⍴ √(V(T1 ) xV( T2 ))} / 4
            = { V(T1 ) +  ⍴ V(T1 ) } / 2
            = { V(T1 )  (1+ ⍴)} / 2          ...............2
where   ⍴  is the Karl Pearson coefficient of correlation between T1 & T2 .
Since T1 is minimum variance unbiased estimator of  θ.
then 
 V(T1 ) ≤ V( T ) 
i.e.   V(T1 ) ≤ { V(T1 )  (1+ ⍴)} / 2   
      V(T1 ) ≤ { V(T1 )  (1+ ⍴)} / 2   
 1≤  (1+ ⍴) / 2   
 2 ≤  (1+ ⍴)   
 1≤  ⍴ 
we have |⍴ | ≤  1 therefore we must have ⍴  = 1.
i.e.   T1 & T2 have linear relationship of the form 
 T1  = ɑ + 𝜷T2.........3
Where  ɑ and  𝜷 are constant independent on parameterθ
E( T1 ) = E(ɑ + 𝜷T2)
E( T1 ) = E(ɑ )+ E(𝜷T2)
θ   = ɑ + 𝜷θ. ........4
now variance 

 V(T1  )=V( ɑ + 𝜷T2)
   V(T1  )=V( 𝜷T2)  
  V(T1  )=𝜷²V(T2)
therefore   𝜷² = 1
𝜷=+-1
𝜷 =1 ................5
because ⍴  = 1. T1 & T2 are positive 
from equation 4 and 5 we get
θ   = ɑ + θ
 ɑ = 0
putting 𝜷 =1,ɑ = 0 in equation 3
 T1  = 0 + 1xT2
     T1  = T2 
Thus if UMVUE exist, it is unique.

III. Consistency:

 

                


 
 






Labels

Labels:

Comments

Post a Comment

Popular posts from this blog

MCQ'S based on Basic Statistics (For B. Com. II Business Statistics)

    (MCQ Based on Probability, Index Number, Time Series   and Statistical Quality Control Sem - IV)                                                            1.The control chart were developed by ……         A) Karl Pearson B) R.A. fisher C) W.A. Shewhart D) B. Benjamin   2.the mean = 4 and variance = 2 for binomial r.v. x then value of n is….. A) 7 B) 10 C) 8 D)9   3.the mean = 3 and variance = 2 for binomial r.v. x then value of n is….. A) 7 B) 10 C) 8 D)9 4. If sampl...
Post a Comment
Read more

Measures of Central Tendency :Mean, Median and Mode

Changing Color Blog Name  Measures of Central Tendency  I. Introduction. II. Requirements of good measures. III. Mean Definition. IV . Properties  V. Merits and Demerits. VI. Examples VII.  Weighted Arithmetic Mean VIII. Median IX. Quartiles I. Introduction Everybody is familiar with the word Average. and everybody are used the word average in daily life as, average marks, average of bike, average speed etc. In real life the average is used to represent the whole data, or it is a single figure is represent the whole data. the average value is lies around the centre of the data. consider the example if we are interested to measure the height of the all student and remember the heights of all student, in that case there are 2700 students then it is not possible to remember the all 2700 students height so we find out the one value that represent the height of the all 2700 students in college. therefore the single value represent ...
1 comment
Read more

Business Statistics Notes ( Meaning, Scope, Limitations of statistics and sampling Methods)

  Business Statistics Paper I Notes. Welcome to our comprehensive collection of notes for the Business Statistics!  my aim is to provided you  with the knowledge you need as you begin your journey to comprehend the essential ideas of this subject. Statistics is a science of collecting, Presenting, analyzing, interpreting data to make informed business decisions. It forms the backbone of modern-day business practices, guiding organizations in optimizing processes, identifying trends, and predicting outcomes. I will explore several important topics through these notes, such as: 1. Introduction to Statistics. :  meaning definition and scope of  Statistics. 2. Data collection methods. 3. Sampling techniques. 4. Measures of  central tendency : Mean, Median, Mode. 5. Measures of Dispersion : Relative and Absolute Measures of dispersion,  Range, Q.D., Standard deviation, Variance. coefficient of variation.  6.Analysis of bivariate data: Correlation, Regr...
Post a Comment
Read more

Classification, Tabulation, Frequency Distribution, Diagrams & Graphical Presentation.

Business Statistics I    Classification, Tabulation, Frequency Distribution ,  Diagrams & Graphical Presentation. In this section we study the following point : i. Classification and it types. ii. Tabulation. iii. Frequency and Frequency Distribution. iv. Some important concepts. v. Diagrams & Graphical Presentation   I. Classification and it's types:        Classification:- The process of arranging data into different classes or groups according to their common  characteristics is called classification. e.g. we dividing students into age, gender and religion. It is a classification of students into age, gender and religion.  Or  Classification is a method used to categorize data into different groups based on the values of specific variable.  The purpose of classification is to condenses the data, simplifies complexities, it useful to comparison and helps to analysis. The following are some criteria to classi...
Post a Comment
Read more

Measures of Dispersion : Range , Quartile Deviation, Standard Deviation and Variance.

Measures of Dispersion :  I.  Introduction. II. Requirements of good measures. III. Uses of Measures of Dispersion. IV.  Methods Of Studying Dispersion:     i.  Absolute Measures of Dispersions :             i. Range (R)          ii. Quartile Deviation (Q.D.)          iii. Mean Deviation (M.D.)         iv. Standard Deviation (S. D.)         v. Variance    ii.   Relative Measures of Dispersions :              i. Coefficient of Range          ii. Coefficient of Quartile Deviation (Q.D.)          iii. Coefficient of Mean Deviation (M.D.)         iv. Coefficient of Standard Deviation (S. D.)         v. Coefficien...
Post a Comment
Read more

Basic Concepts of Probability and Binomial Distribution , Poisson Distribution.

 Probability:  Basic concepts of Probability:  Probability is a way to measure hoe likely something is to happen. Probability is number between 0 and 1, where probability is 0 means is not happen at all and probability is 1 means it will be definitely happen, e.g. if we tossed coin there is a 50% chance to get head and 50% chance to get tail, it can be represented in probability as 0.5 for each outcome to get head and tail. Probability is used to help us taking decision and predicting the likelihood of the event in many areas, that are science, finance and Statistics.  Now we learn the some basic concepts that used in Probability:  i) Random Experiment OR Trail: A Random Experiment is an process that get one or more possible outcomes. examples of random experiment include tossing a coin, rolling a die, drawing  a card from pack of card etc. using this we specify the possible outcomes known as sample pace.  ii)Outcome: An outcome is a result of experi...
1 comment
Read more

The Power of Statistics: A Gateway to Exciting Opportunities

  My Blog The Power of Statistics: A Gateway to Exciting Opportunities     Hey there, future statistician! Ever wondered how Netflix seems to know exactly what shows you'll love, how sports teams break down player performance, or how businesses figure out their pricing strategies? The answer is statistics—a fascinating field that helps us make sense of data in our everyday lives. Let's dive into why choosing statistics for your B.Sc. Part First can lead you to some exciting opportunities.     Why Statistics Matters in Everyday Life     From predicting election outcomes and analyzing social media trends to understanding consumer behavior and optimizing public transport routes, statistics are crucial. It's the backbone of modern decision-making, helping us sift through complex data to uncover meaningful insights that drive innovation and progress.   The Role of Statistics in Future Opportunities ...
Post a Comment
Read more

Statistical Inference I ( Theory of Estimation) : Unbiased it's properties and examples

 📚Statistical Inference I Notes The theory of  estimation invented by Prof. R. A. Fisher in a series of fundamental papers in around 1930. Statistical inference is a process of drawing conclusions about a population based on the information gathered from a sample. It involves using statistical techniques to analyse data, estimate parameters, test hypotheses, and quantify uncertainty. In essence, it allows us to make inferences about a larger group (i.e. population) based on the characteristics observed in a smaller subset (i.e. sample) of that group. Notation of parameter: Let x be a random variable having distribution function F or f is a population distribution. the constant of  distribution function of F is known as Parameter. In general the parameter is denoted as any Greek Letters as θ.   now we see the some basic terms :  i. Population : in a statistics, The group of individual under study is called Population. the population is may be a group of obj...
Post a Comment
Read more