Skip to main content

Statistical Inference I ( Theory of Estimation) : Unbiased it's properties and examples

 📚Statistical Inference I Notes


The theory of  estimation invented by Prof. R. A. Fisher in a series of fundamental papers in around 1930.

Statistical inference is a process of drawing conclusions about a population based on the information gathered from a sample. It involves using statistical techniques to analyse data, estimate parameters, test hypotheses, and quantify uncertainty. In essence, it allows us to make inferences about a larger group (i.e. population) based on the characteristics observed in a smaller subset (i.e. sample) of that group.

Notation of parameter: Let x be a random variable having distribution function F or f is a population distribution. the constant of  distribution function of F is known as Parameter. In general the parameter is denoted as any Greek Letters as θ.  

now we see the some basic terms : 

i. Population : in a statistics, The group of individual under study is called Population. the population is may be a group of object, animate like persons or inanimate like group of non-living cars. e.g. if we are interested to study the economic condition of males in Sangli district then the all males in Sangli district are the population. 

ii. Sample : A sub-group of a population is called sample, the sample is a portion of a population which is examine to estimate a characteristic of population. but selected sample may be a true representative of the population. 

iii. Parameter: Parameter is a constant value based on the population observation. they are usually denoted as Greek Letters Like θ, 𝝑,  𝝁, 𝞂. e.g. binomial distribution has parameter P. normal distribution has parameters are  𝝁, 𝞂. but these are notations we use any notation for represent the parameter of any distribution.

iv. Parameter Space: The set of all values of parameter θ is called as Parameter Space ad it is denoted as Θ. and the symbol Θ is read as script theta. e.g. X has normal distribution with mean 𝝻 and variance 𝞼² then the parameter space is  Θ = { (𝝻, 𝞼² ) : -∞ <  𝝻  < ∞ ; 𝞼² > 0}

in particular  if the variance 𝞼² = 1 then parameter space is  Θ = { (𝝻, 1 ) : -∞ <  𝝻  < ∞ }

🔖Point Estimation: 

the random sample of size n drawn form distribution f(x). and the θ be the unknown parameter of the distribution we are interested to finding the value of parameter or estimated value of parameter. but there is problem of point estimation is to choosing the statistics T(X₁, X₂, X₃, .......Xn) that may be consider as the estimate of parameter θ then the statistic  T is said to point estimate of parameter θ if take single value in Θ.  thus an estimator of parameter gives single  value is called a point estimate of parameter. 

Definition: Form sample we obtain single value as estimate of parameter , we call it as point estimate of parameter and the method used to find the estimator is called point estimation or method of estimation.

Estimator

The function of random variable is called estimator, or function of sample observation. it is used to estimate the parameter. 

Estimate: the numerical value of estimator is called estimate. 

Standard Error: 

Standard error of estimator is the positive square root of variance of estimator.

Properties of estimators: 

i. Unbiasedness 

ii. Efficiency

iii. Consistency 

iv. Sufficiency

we see one by one properties of estimator 

i. Unbiasedness

The estimated value of the parameter that false nearest to the true value of parameter, then this property of estimator is called as unbiasedness.

Definition: An estimator T =T(X₁, X₂, X₃, .......Xn) is said to be an Unbiased estimator of parameter θ,  if the E (T) = θ  ;∀ θ ɛ Θ.

thus the unbiasedness means essentially the average value of estimate that will close to the true Parameter value. i.e. if we were to take sample of size n and for each sample compute the observed value of T =T(X₁, X₂, X₃, .......Xn)

then E (T) = θ  ;∀ θ ɛ Θ.

    a) Biased Estimator: 

Definition :  An estimator T =T(X₁, X₂, X₃, .......Xn) is said to be an biased estimator of parameter θ,  if the E (T) ≠ θ  ;∀ θ ɛ Θ. therefore this quantity is

 b(T, θ) = E(T- θ)

 b(T, θ) =  E(T) - θ    is called biased estimator of T.

there are two type of biases: 

i. Positive Bias  and ii. Negative Bias

    i. Positive Bias : If the Bias is greater than zero then this bias is called Positive Bias.

i.e. b(T, θ) = E(T- θ)  > 0

i.e. E(T) > θ

    ii. Negative Bias : If the Bias is greater than zero then this bias is called Negative Bias.

i.e. b(T, θ) = E(T- θ)  < 0

i.e. E(T) < θ

Examples

    1. Let X₁, X₂, X₃, .......Xn be a random sample of size n from distribution with finite mean μ then show that the sample mean is unbiased estimator of μ.

SolutionLet X₁, X₂, X₃, .......Xn be a random sample of size n from distribution with finite mean μ.

therefore E(x) = μ

by definition of unbiased estimator 

i.e. E (T) = θ

consider T = sample mean  = x̄ = (1/n) ∑Xi

now E(T ) = E( x̄ ) = E ((1/n) ∑Xi)

                        = (1/n) x n x E ( X )

                        = E ( X ) 

                        = μ

E( ) = μ

therefore the sample mean ( i.e. x̄ )  is  an unbiased estimator of population mean μ.

    2. Let X₁, X₂, X₃, .......Xn a  ramdom sample from normal distribution with mean μ and variance 1. then Show that T =  (1/n) ∑Xis an biased estimator  of  μ² + 1.

Solution: 

Let x has normal distribution with mean ( μ, 1).

Then E(x) = μ and v(x) = 1

by definition of unbiased estimator 

i.e.  E (T) = θ

consider T = (1/n) ∑X

now E(T ) = E [(1/n) ∑X²i ]

                       = (1/n) x n x E(X²i )  ..................1

We know that V (X) = E (X²)  - [E(X)]²

therefore  E(X²i ) = V(X) +  [E(X)]²

  E(X²i ) = 1 + [μ ]²

E(X²i ) = 1 + μ²

put  E(X²i ) = 1 + μ²   in equation 1

E [(1/n) ∑X²i ](1/n) x n x E(X²i 

                   = 1 + μ²   

E [(1/n) ∑X²i ] 1 + μ² 

therefore   (1/n) ∑X²i  is an  unbiased estimator of  1 + μ² .


Properties of Unbiasedness.

I) If T is an unbiased estimator of  𝛉 then Ø(T) is an Unbiased estimator of  Ø(𝛉). Provided Ø(.)  is a linear function.

Proof : Here, Given that  T is an unbiased estimator of  𝛉 

i.e. E (T) = 𝛉

and Ø(.) is an linear function 

Consider a and b are two constant then 

Ø(T) = aT +b is a linear function of T

therefore,  E[Ø(T)] = E (aT +b)

                                = a E(T) +b

                                =  a𝛉 + b     it is a linear function of Ø(𝛉).

                                 =Ø(𝛉)

 E[Ø(T)] = Ø(𝛉)

Hence Ø(T) is an unbiased estimator of Ø(𝛉).

If T is an unbiased estimator of 𝛉 then Ø(T) is an Unbiased estimator of  Ø(𝛉). Provided Ø(.)  is a linear function.

this property is not hold when Ø(.) is non-liner function.


II. Two distinct unbiased estimators of  Ø(𝛉) gives rise to infinitely many unbiased estimators of  Ø(𝛉).

Proof:

Let T and T2 are Two distinct unbiased estimators of  parametric function  Ø(𝛉) based on random sample X₁, X₂, X₃, .......Xn

i.e. E(T) =E ( T2) =  Ø(𝛉);  ∀ θ ɛ Θ.

Let us consider a linear combination of these two estimators,  T and T2  of   Ø(𝛉) as

T𝛼 = 𝛼  T + (1- 𝛼)  T2      for any real value of   𝛼 ɛ R

Now E(T𝛼 ) = E [𝛼  T + (1- 𝛼)  T2 ]

                    = 𝛼 E( T1) + (1- 𝛼) E( T2 )

                   = 𝛼 Ø(𝛉) + (1- 𝛼) Ø(𝛉)

                     = 𝛼 Ø(𝛉) + Ø(𝛉)𝛼 Ø(𝛉)

                    = Ø(𝛉)

 E(T𝛼 ) Ø(𝛉) 

Therefore T𝛼  is an unbiased estimator of Ø(𝛉)  for any real value of 𝛼.

 we take any real value for  𝛼  we get infinity many unbiased estimators.


Example:  3. If T is unbiased estimator of 𝛉, then show that T² is a biased estimator of 𝛉².

Solution: given that T is unbiased estimator of 𝛉 then E(T) =  𝛉

we have variance of T = Var(T) = E(T²) - [E(T)]²

                                                    = E(T²) - [𝛉

therefore  E(T²) = Var(T) +  𝛉²   we have variance of T is greater than 0 i.e. Var(T)>0

    E(T²) > 𝛉²  means  E(T²) is not same as  𝛉² the value of 𝛉² is greater than E(T²)

i.e. E(T²)  𝛉²  this is the definition of biased estimator.

hence T² is a biased estimator of 𝛉².


4. Let X₁, X₂, X₃, .......Xn a  ramdom sample from Poisson distribution with parameter 𝛉 then show that T= ∝ x̄ + (1-∝) s² is unbiased estimator of  𝛉 for any real value of ∝. given that  x̄  and s² are unbiased estimators of parameter 𝛉.

Solution: We know that sample mean x̄  and   sample mean square are unbiased estimator of  𝛉.

E (x̄ ) =E(s²) = 𝛉

now  T= ∝ x̄ + (1-∝) s² 

taking expectation 

E(T)= E(∝ x̄ + (1-∝) s² )

E(T)= ∝E( x)̄ + (1-∝)E( s² )

E(T)= ∝ 𝛉̄ + (1-∝) 𝛉

E(T) = ∝ 𝛉̄ +  𝛉 𝛉

E(T) = 𝛉

hence T is unbiased estimator of 𝛉. for any real value of  ∝  hence we get infinity many unbiased estimators of parameter 𝛉.




 

 











Comments

Popular posts from this blog

MCQ'S based on Basic Statistics (For B. Com. II Business Statistics)

    (MCQ Based on Probability, Index Number, Time Series   and Statistical Quality Control Sem - IV)                                                            1.The control chart were developed by ……         A) Karl Pearson B) R.A. fisher C) W.A. Shewhart D) B. Benjamin   2.the mean = 4 and variance = 2 for binomial r.v. x then value of n is….. A) 7 B) 10 C) 8 D)9   3.the mean = 3 and variance = 2 for binomial r.v. x then value of n is….. A) 7 B) 10 C) 8 D)9 4. If sampl...

Basic Concepts of Probability and Binomial Distribution , Poisson Distribution.

 Probability:  Basic concepts of Probability:  Probability is a way to measure hoe likely something is to happen. Probability is number between 0 and 1, where probability is 0 means is not happen at all and probability is 1 means it will be definitely happen, e.g. if we tossed coin there is a 50% chance to get head and 50% chance to get tail, it can be represented in probability as 0.5 for each outcome to get head and tail. Probability is used to help us taking decision and predicting the likelihood of the event in many areas, that are science, finance and Statistics.  Now we learn the some basic concepts that used in Probability:  i) Random Experiment OR Trail: A Random Experiment is an process that get one or more possible outcomes. examples of random experiment include tossing a coin, rolling a die, drawing  a card from pack of card etc. using this we specify the possible outcomes known as sample pace.  ii)Outcome: An outcome is a result of experi...

Statistical Inference: Basic Terms and Definitions.

  📚📖 Statistical Inference: Basic Terms. The theory of estimation is of paramount importance in statistics for several reasons. Firstly, it allows researchers to make informed inferences about population characteristics based on limited sample data. Since it is often impractical or impossible to measure an entire population, estimation provides a framework to generalize findings from a sample to the larger population. By employing various estimation methods, statisticians can estimate population parameters such as means, proportions, and variances, providing valuable insights into the population's characteristics. Second, the theory of estimating aids in quantifying the estimates' inherent uncertainty. Measures like standard errors, confidence intervals, and p-values are included with estimators to provide  an idea of how accurate and reliable the estimates are. The range of possible values for the population characteristics and the degree of confidence attached to those est...

B. Com. I Practical No. 3 :Diagrammatic representation of data by using Pie Diagram and Bar Diagrams.

Practical No. 3 :Diagrammatic representation of data by using Pie Diagram and Bar Diagrams. Diagrammatic Presentation. We have observed the classification and tabulation method. We use this method to take a lot of information and make it fit into a small table. The reason we do this is to make the information more organized and easier to understand. Tabulation helps us arrange data neatly so that it's not messy and confusing. tabulation is a way to make big files of information look neat and tidy in a table.  but better and beautiful way to represent data using diagrams and graphs. the diagram and graph have some advantages because that used to visualise the data. that helps to understand and give information easily to any common man or any one, following are the some  advantages of diagram and graph.  I. Advantages i. Data Representation: Diagrams and graphs are excellent for presenting data visually, making trends, comparisons, and statistical information easier to...

Index Number

 Index Number      Introduction  We seen in measures of central tendency the data can be reduced to a single figure by calculating an average and two series can be compared by their averages. But the data are homogeneous then the average is meaningful. (Data is homogeneous means data in same type). If the two series of the price of commodity for two years. It is clear that we cannot compare the cost of living for two years by using simple average of the price of the commodities. For that type of problem we need type of average is called Index number. Index number firstly defined or developed to study the effect of price change on the cost of living. But now days the theory of index number is extended to the field of wholesale price, industrial production, agricultural production etc. Index number is like barometers to measure the change in change in economics activities.   An index may be defined as a " specialized  average designed to measure the...

Method of Moment & Maximum Likelihood Estimator: Method, Properties and Examples.

 Statistical Inference I: Method Of Moment:   One of the oldest method of finding estimator is Method of Moment, it was discovered by Karl Pearson in 1884.  Method of Moment Estimator Let X1, X2, ........Xn be a random sample from a population with probability density function (pdf) f(x, θ) or probability mass function (pmf) p(x) with parameters θ1, θ2,……..θk. If μ r ' (r-th raw moment about the origin) then μ r ' = ∫ -∞ ∞ x r f(x,θ) dx for r=1,2,3,….k .........Equation i In general, μ 1 ' , μ 2 ' ,…..μ k ' will be functions of parameters θ 1 , θ 2 ,……..θ k . Let X 1 , X 2 ,……X n be the random sample of size n from the population. The method of moments consists of solving "k" equations (in Equation i) for θ 1 , θ 2 ,……..θ k to obtain estimators for the parameters by equating μ 1 ' , μ 2 ' ,…..μ k ' with the corresponding sample moments m 1 ' , m 2 ' ,…..m k ' . Where m r ' = sample m...

B. Com. I (OE) Basic Statistics Practical 6. Simple random sampling (with and without replacement) and stratified random sampling.

  DEPARTMENT OF STATISTICS B. Com. I: Practical - I Expt. No. 6                                                                                        Date:     /     / 2024 Title: Sampling: Simple random sampling (with and without replacement) and stratified random sampling. Q.1 In population of size N=20, the observation were 4, 7, 10, 14, 17, 18, 20, 24, 25, 27, 29, 30, 34, 37, 39, 40, 42, 45, 47, 48. Draw Random Sample of size 5 form given population using SRSWOR. Q.2 In population of size N= 10, the observation ...

B. Com. -I Statistics Practical No. 1 Classification, tabulation and frequency distribution –I: Qualitative data.

  Shree GaneshA B. Com. Part – I: Semester – I OE–I    Semester – I (BASIC STATISTICS PRACTICAL-I) Practical: 60 Hrs. Marks: 50 (Credits: 02) Course Outcomes: After completion of this practical course, the student will be able to: i) apply sampling techniques in real life. ii) perform classification and tabulation of primary data. iii) represent the data by means of simple diagrams and graphs. iv) summarize data by computing measures of central tendency.   LIST OF PRACTICALS: 1. Classification, tabulation and frequency distribution –I: Qualitative data. 2. Classification, tabulation and frequency distribution –II : Quantitative data. 3. Diagrammatic representation of data by using Pie Diagram and Bar Diagrams. 4. Graphical representation of data by using Histogram, Frequency Polygon, Frequency Curve and     Locating Modal Value. 5. Graphical representation of data by using Ogive Curves and Locating Quartile Values....

Time Series

 Time series  Introduction:-         We see the many variables are changes over period of time that are population (I.e. population are changes over time means population increase day by day), monthly demand of commodity, food production, agriculture production increases and that can be observed over period of times known as time series. Time series is defined as a set of observation arranged according to time is called time series. Or a time Series is a set of statistical observation arnging chronological order. ( Chronological order means it is arrangements of variable according to time) and it gives information about variable.  Also we draw the graph of time series to see the behaviour of variable over time. It can be used of forecasting. The analysis of time series is helpful to economist, business men, also for scientist etc. Because it used to forecasting the future, observing the past behaviour of that variable or items. Also planning for future...