Mdthbs

My question is simple as that: what is joint estimation? And what does it mean in the context of regression analysis? How is it done? I wandered in the mighty Internet for quite some time but did not find answers to these questions.

Joint estimation is, simply, jointly estimating two (or more) things at the same time. It can be as simple as estimating the mean and standard deviation from a sample.

In a lot of the literature, the term is invoked because a special estimating procedure has to be used. This is usually the case when one quantity depends on the other and vice versa so that an analytic solution to the problem is intractable. How exactly joint estimation is done depends entirely on the problem.

One method that pops up often for "joint modeling" or joint estimation is the EM-algorithm. EM stands for expectation - maximization. By alternating these steps, the E-step fills in the missing data that otherwise depend on component A, and the M-step finds optimal estimates for component B. By iterating the E and M steps, you can find a maximum likelihood estimate of A and B, thus jointly estimate these things.

In a statistical context, the term "joint estimation" could conceivably mean one of two things:

• 
  
  1. The simultaneous estimation of two scalar parameters (or equivalently, the estimation of a vector parameter with at least two elements); or
  
  


• 
  
  2. The estimation of a single parameter pertaining to a joint (e.g., in the study of carpentry, plumbing systems, or marijuana).
  
  

Of those two options, the second one is a joke, so almost certainly, joint estimation refers to simultaneously estimating two scalar parameters at once.

Joint estimation is using data to estimate two or more parameters at the same time. Separate estimation evaluates each parameter one at a time.

Estimation is the result of some form of optimization process. Because of this, there do not exist unique estimation solutions in statistics. If you change your goal, then you change what is optimal. When you first learn things such as regression, no one tells you why you are doing what you are doing. The goal of the instructor is to give you a degree of basic functionality using methods that work in a wide range of circumstances. At the beginning, you are not learning about regression. Instead, you are learning one or two regression methods that are widely applicable in a wide range of circumstances.

The fact you are looking for solutions that solve a hidden goal makes it a bit difficult to understand.

In the context of regression, imagine the following algebraic expression is true $$z=beta_xx+beta_yy+alpha$$. A truism in statistics is the more information that you have, the better off you are. Let us assume that you need to determine what values for $z$ will happen when you see $(x,y)$. The problem is that you do not know the true values for ${beta_x,beta_y,alpha}$. You have a large, complete data set of ${x,y,z}$.

In separate estimation, you would estimate one parameter at a time. In joint estimation, you would estimate all of them at once.

As a rule of thumb, joint estimation is more accurate than a separate estimate with a large complete data set. There is one general exception to that. Imagine you have a large set of $x$ and $z$ but a small set of $y$. Imagine most of your $y$ values are missing.

In many estimation routines, you would delete the missing $x$s and $z$s and reduce down the set you are working from until all sets are complete. If you have deleted enough data, it can be more accurate to use the large number of $x$s and $z$s separately to estimate $z=beta_xx+alpha$ and $z=beta_yy+alpha$ than together.

Now as to how it is done. All estimation, excluding a few exceptional cases, uses calculus to find an estimator that minimizes some form of loss or some type of risk. The concern is that you will be unlucky in choosing your sample. Unfortunately, there is an infinite number of loss functions. There is also an infinite number of risk functions.

I found several videos for you because it is a giant topic so that you can look at it in a more general form. They are from Mathematical Monk.

https://www.youtube.com/watch?v=6GhSiM0frIk

https://www.youtube.com/watch?v=5SPm4TmYTX0

https://www.youtube.com/watch?v=b1GxZdFN6cY

and

https://www.youtube.com/watch?v=WdnP1gmb8Hw.