Skip to main content

Statistical Inference I ( Theory of Estimation) : Unbiased it's properties and examples

 📚Statistical Inference I Notes


The theory of  estimation invented by Prof. R. A. Fisher in a series of fundamental papers in around 1930.

Statistical inference is a process of drawing conclusions about a population based on the information gathered from a sample. It involves using statistical techniques to analyse data, estimate parameters, test hypotheses, and quantify uncertainty. In essence, it allows us to make inferences about a larger group (i.e. population) based on the characteristics observed in a smaller subset (i.e. sample) of that group.

Notation of parameter: Let x be a random variable having distribution function F or f is a population distribution. the constant of  distribution function of F is known as Parameter. In general the parameter is denoted as any Greek Letters as θ.  

now we see the some basic terms : 

i. Population : in a statistics, The group of individual under study is called Population. the population is may be a group of object, animate like persons or inanimate like group of non-living cars. e.g. if we are interested to study the economic condition of males in Sangli district then the all males in Sangli district are the population. 

ii. Sample : A sub-group of a population is called sample, the sample is a portion of a population which is examine to estimate a characteristic of population. but selected sample may be a true representative of the population. 

iii. Parameter: Parameter is a constant value based on the population observation. they are usually denoted as Greek Letters Like θ, 𝝑,  𝝁, 𝞂. e.g. binomial distribution has parameter P. normal distribution has parameters are  𝝁, 𝞂. but these are notations we use any notation for represent the parameter of any distribution.

iv. Parameter Space: The set of all values of parameter θ is called as Parameter Space ad it is denoted as Θ. and the symbol Θ is read as script theta. e.g. X has normal distribution with mean 𝝻 and variance 𝞼² then the parameter space is  Θ = { (𝝻, 𝞼² ) : -∞ <  𝝻  < ∞ ; 𝞼² > 0}

in particular  if the variance 𝞼² = 1 then parameter space is  Θ = { (𝝻, 1 ) : -∞ <  𝝻  < ∞ }

🔖Point Estimation: 

the random sample of size n drawn form distribution f(x). and the θ be the unknown parameter of the distribution we are interested to finding the value of parameter or estimated value of parameter. but there is problem of point estimation is to choosing the statistics T(X₁, X₂, X₃, .......Xn) that may be consider as the estimate of parameter θ then the statistic  T is said to point estimate of parameter θ if take single value in Θ.  thus an estimator of parameter gives single  value is called a point estimate of parameter. 

Definition: Form sample we obtain single value as estimate of parameter , we call it as point estimate of parameter and the method used to find the estimator is called point estimation or method of estimation.

Estimator

The function of random variable is called estimator, or function of sample observation. it is used to estimate the parameter. 

Estimate: the numerical value of estimator is called estimate. 

Standard Error: 

Standard error of estimator is the positive square root of variance of estimator.

Properties of estimators: 

i. Unbiasedness 

ii. Efficiency

iii. Consistency 

iv. Sufficiency

we see one by one properties of estimator 

i. Unbiasedness

The estimated value of the parameter that false nearest to the true value of parameter, then this property of estimator is called as unbiasedness.

Definition: An estimator T =T(X₁, X₂, X₃, .......Xn) is said to be an Unbiased estimator of parameter θ,  if the E (T) = θ  ;∀ θ ɛ Θ.

thus the unbiasedness means essentially the average value of estimate that will close to the true Parameter value. i.e. if we were to take sample of size n and for each sample compute the observed value of T =T(X₁, X₂, X₃, .......Xn)

then E (T) = θ  ;∀ θ ɛ Θ.

    a) Biased Estimator: 

Definition :  An estimator T =T(X₁, X₂, X₃, .......Xn) is said to be an biased estimator of parameter θ,  if the E (T) ≠ θ  ;∀ θ ɛ Θ. therefore this quantity is

 b(T, θ) = E(T- θ)

 b(T, θ) =  E(T) - θ    is called biased estimator of T.

there are two type of biases: 

i. Positive Bias  and ii. Negative Bias

    i. Positive Bias : If the Bias is greater than zero then this bias is called Positive Bias.

i.e. b(T, θ) = E(T- θ)  > 0

i.e. E(T) > θ

    ii. Negative Bias : If the Bias is greater than zero then this bias is called Negative Bias.

i.e. b(T, θ) = E(T- θ)  < 0

i.e. E(T) < θ

Examples

    1. Let X₁, X₂, X₃, .......Xn be a random sample of size n from distribution with finite mean μ then show that the sample mean is unbiased estimator of μ.

SolutionLet X₁, X₂, X₃, .......Xn be a random sample of size n from distribution with finite mean μ.

therefore E(x) = μ

by definition of unbiased estimator 

i.e. E (T) = θ

consider T = sample mean  = x̄ = (1/n) ∑Xi

now E(T ) = E( x̄ ) = E ((1/n) ∑Xi)

                        = (1/n) x n x E ( X )

                        = E ( X ) 

                        = μ

E( ) = μ

therefore the sample mean ( i.e. x̄ )  is  an unbiased estimator of population mean μ.

    2. Let X₁, X₂, X₃, .......Xn a  ramdom sample from normal distribution with mean μ and variance 1. then Show that T =  (1/n) ∑Xis an biased estimator  of  μ² + 1.

Solution: 

Let x has normal distribution with mean ( μ, 1).

Then E(x) = μ and v(x) = 1

by definition of unbiased estimator 

i.e.  E (T) = θ

consider T = (1/n) ∑X

now E(T ) = E [(1/n) ∑X²i ]

                       = (1/n) x n x E(X²i )  ..................1

We know that V (X) = E (X²)  - [E(X)]²

therefore  E(X²i ) = V(X) +  [E(X)]²

  E(X²i ) = 1 + [μ ]²

E(X²i ) = 1 + μ²

put  E(X²i ) = 1 + μ²   in equation 1

E [(1/n) ∑X²i ](1/n) x n x E(X²i 

                   = 1 + μ²   

E [(1/n) ∑X²i ] 1 + μ² 

therefore   (1/n) ∑X²i  is an  unbiased estimator of  1 + μ² .


Properties of Unbiasedness.

I) If T is an unbiased estimator of  𝛉 then Ø(T) is an Unbiased estimator of  Ø(𝛉). Provided Ø(.)  is a linear function.

Proof : Here, Given that  T is an unbiased estimator of  𝛉 

i.e. E (T) = 𝛉

and Ø(.) is an linear function 

Consider a and b are two constant then 

Ø(T) = aT +b is a linear function of T

therefore,  E[Ø(T)] = E (aT +b)

                                = a E(T) +b

                                =  a𝛉 + b     it is a linear function of Ø(𝛉).

                                 =Ø(𝛉)

 E[Ø(T)] = Ø(𝛉)

Hence Ø(T) is an unbiased estimator of Ø(𝛉).

If T is an unbiased estimator of 𝛉 then Ø(T) is an Unbiased estimator of  Ø(𝛉). Provided Ø(.)  is a linear function.

this property is not hold when Ø(.) is non-liner function.


II. Two distinct unbiased estimators of  Ø(𝛉) gives rise to infinitely many unbiased estimators of  Ø(𝛉).

Proof:

Let T and T2 are Two distinct unbiased estimators of  parametric function  Ø(𝛉) based on random sample X₁, X₂, X₃, .......Xn

i.e. E(T) =E ( T2) =  Ø(𝛉);  ∀ θ ɛ Θ.

Let us consider a linear combination of these two estimators,  T and T2  of   Ø(𝛉) as

T𝛼 = 𝛼  T + (1- 𝛼)  T2      for any real value of   𝛼 ɛ R

Now E(T𝛼 ) = E [𝛼  T + (1- 𝛼)  T2 ]

                    = 𝛼 E( T1) + (1- 𝛼) E( T2 )

                   = 𝛼 Ø(𝛉) + (1- 𝛼) Ø(𝛉)

                     = 𝛼 Ø(𝛉) + Ø(𝛉)𝛼 Ø(𝛉)

                    = Ø(𝛉)

 E(T𝛼 ) Ø(𝛉) 

Therefore T𝛼  is an unbiased estimator of Ø(𝛉)  for any real value of 𝛼.

 we take any real value for  𝛼  we get infinity many unbiased estimators.


Example:  3. If T is unbiased estimator of 𝛉, then show that T² is a biased estimator of 𝛉².

Solution: given that T is unbiased estimator of 𝛉 then E(T) =  𝛉

we have variance of T = Var(T) = E(T²) - [E(T)]²

                                                    = E(T²) - [𝛉

therefore  E(T²) = Var(T) +  𝛉²   we have variance of T is greater than 0 i.e. Var(T)>0

    E(T²) > 𝛉²  means  E(T²) is not same as  𝛉² the value of 𝛉² is greater than E(T²)

i.e. E(T²)  𝛉²  this is the definition of biased estimator.

hence T² is a biased estimator of 𝛉².


4. Let X₁, X₂, X₃, .......Xn a  ramdom sample from Poisson distribution with parameter 𝛉 then show that T= ∝ x̄ + (1-∝) s² is unbiased estimator of  𝛉 for any real value of ∝. given that  x̄  and s² are unbiased estimators of parameter 𝛉.

Solution: We know that sample mean x̄  and   sample mean square are unbiased estimator of  𝛉.

E (x̄ ) =E(s²) = 𝛉

now  T= ∝ x̄ + (1-∝) s² 

taking expectation 

E(T)= E(∝ x̄ + (1-∝) s² )

E(T)= ∝E( x)̄ + (1-∝)E( s² )

E(T)= ∝ 𝛉̄ + (1-∝) 𝛉

E(T) = ∝ 𝛉̄ +  𝛉 𝛉

E(T) = 𝛉

hence T is unbiased estimator of 𝛉. for any real value of  ∝  hence we get infinity many unbiased estimators of parameter 𝛉.




 

 











Comments

Popular posts from this blog

MCQ'S based on Basic Statistics (For B. Com. II Business Statistics)

    (MCQ Based on Probability, Index Number, Time Series   and Statistical Quality Control Sem - IV)                                                            1.The control chart were developed by ……         A) Karl Pearson B) R.A. fisher C) W.A. Shewhart D) B. Benjamin   2.the mean = 4 and variance = 2 for binomial r.v. x then value of n is….. A) 7 B) 10 C) 8 D)9   3.the mean = 3 and variance = 2 for binomial r.v. x then value of n is….. A) 7 B) 10 C) 8 D)9 4. If sampl...

Measures of Central Tendency :Mean, Median and Mode

Changing Color Blog Name  Measures of Central Tendency  I. Introduction. II. Requirements of good measures. III. Mean Definition. IV . Properties  V. Merits and Demerits. VI. Examples VII.  Weighted Arithmetic Mean VIII. Median IX. Quartiles I. Introduction Everybody is familiar with the word Average. and everybody are used the word average in daily life as, average marks, average of bike, average speed etc. In real life the average is used to represent the whole data, or it is a single figure is represent the whole data. the average value is lies around the centre of the data. consider the example if we are interested to measure the height of the all student and remember the heights of all student, in that case there are 2700 students then it is not possible to remember the all 2700 students height so we find out the one value that represent the height of the all 2700 students in college. therefore the single value represent ...

Business Statistics Notes ( Meaning, Scope, Limitations of statistics and sampling Methods)

  Business Statistics Paper I Notes. Welcome to our comprehensive collection of notes for the Business Statistics!  my aim is to provided you  with the knowledge you need as you begin your journey to comprehend the essential ideas of this subject. Statistics is a science of collecting, Presenting, analyzing, interpreting data to make informed business decisions. It forms the backbone of modern-day business practices, guiding organizations in optimizing processes, identifying trends, and predicting outcomes. I will explore several important topics through these notes, such as: 1. Introduction to Statistics. :  meaning definition and scope of  Statistics. 2. Data collection methods. 3. Sampling techniques. 4. Measures of  central tendency : Mean, Median, Mode. 5. Measures of Dispersion : Relative and Absolute Measures of dispersion,  Range, Q.D., Standard deviation, Variance. coefficient of variation.  6.Analysis of bivariate data: Correlation, Regr...

Classification, Tabulation, Frequency Distribution, Diagrams & Graphical Presentation.

Business Statistics I    Classification, Tabulation, Frequency Distribution ,  Diagrams & Graphical Presentation. In this section we study the following point : i. Classification and it types. ii. Tabulation. iii. Frequency and Frequency Distribution. iv. Some important concepts. v. Diagrams & Graphical Presentation   I. Classification and it's types:        Classification:- The process of arranging data into different classes or groups according to their common  characteristics is called classification. e.g. we dividing students into age, gender and religion. It is a classification of students into age, gender and religion.  Or  Classification is a method used to categorize data into different groups based on the values of specific variable.  The purpose of classification is to condenses the data, simplifies complexities, it useful to comparison and helps to analysis. The following are some criteria to classi...

Measures of Dispersion : Range , Quartile Deviation, Standard Deviation and Variance.

Measures of Dispersion :  I.  Introduction. II. Requirements of good measures. III. Uses of Measures of Dispersion. IV.  Methods Of Studying Dispersion:     i.  Absolute Measures of Dispersions :             i. Range (R)          ii. Quartile Deviation (Q.D.)          iii. Mean Deviation (M.D.)         iv. Standard Deviation (S. D.)         v. Variance    ii.   Relative Measures of Dispersions :              i. Coefficient of Range          ii. Coefficient of Quartile Deviation (Q.D.)          iii. Coefficient of Mean Deviation (M.D.)         iv. Coefficient of Standard Deviation (S. D.)         v. Coefficien...

Basic Concepts of Probability and Binomial Distribution , Poisson Distribution.

 Probability:  Basic concepts of Probability:  Probability is a way to measure hoe likely something is to happen. Probability is number between 0 and 1, where probability is 0 means is not happen at all and probability is 1 means it will be definitely happen, e.g. if we tossed coin there is a 50% chance to get head and 50% chance to get tail, it can be represented in probability as 0.5 for each outcome to get head and tail. Probability is used to help us taking decision and predicting the likelihood of the event in many areas, that are science, finance and Statistics.  Now we learn the some basic concepts that used in Probability:  i) Random Experiment OR Trail: A Random Experiment is an process that get one or more possible outcomes. examples of random experiment include tossing a coin, rolling a die, drawing  a card from pack of card etc. using this we specify the possible outcomes known as sample pace.  ii)Outcome: An outcome is a result of experi...

Statistical Inference I ( Theory of estimation : Efficiency)

🔖Statistical Inference I ( Theory of estimation : Efficiency)  In this article we see the  terms:  I. Efficiency. II. Mean Square Error. III. Consistency. 📚 Efficiency:  We know that  two unbiased estimator of parameter gives rise to infinitely many unbiased estimators of parameter. there if one of parameter have two estimators then the problem is to choose one of the best estimator among the class of unbiased estimators. in that case we need to some other criteria to to find out best estimator. therefore, that situation  we check the variability of that estimator, the measure of variability of estimator T around it mean is Var(T). hence If T is an Unbiased estimator of parameter then it's variance gives good precision. the variance is smaller then it give's greater precision. 📑 i. Efficient estimator: An estimator T is said to be an Efficient Estimator of 𝚹, if T is unbiased estimator of    𝛉. and it's variance is less than any other estima...

The Power of Statistics: A Gateway to Exciting Opportunities

  My Blog The Power of Statistics: A Gateway to Exciting Opportunities     Hey there, future statistician! Ever wondered how Netflix seems to know exactly what shows you'll love, how sports teams break down player performance, or how businesses figure out their pricing strategies? The answer is statistics—a fascinating field that helps us make sense of data in our everyday lives. Let's dive into why choosing statistics for your B.Sc. Part First can lead you to some exciting opportunities.     Why Statistics Matters in Everyday Life     From predicting election outcomes and analyzing social media trends to understanding consumer behavior and optimizing public transport routes, statistics are crucial. It's the backbone of modern decision-making, helping us sift through complex data to uncover meaningful insights that drive innovation and progress.   The Role of Statistics in Future Opportunities ...