1. Expectations
    • E[X] = \int xf(x)dx
    • E[c] = c
    • E[X + c] = E[X] + c
    • E[cX] = cE[X]
    • E[X+Y] = E[X] + E[Y]
    • E[XY] = E[X]E[Y] if X and Y are independent
    • E[\vec{X}] = (E[X_1] \ E[X_2] \  ... \  E[X_d])^T
  2. Variance
    • Var(X) = E[(X - E[X])^2]
    • Var(X) = E[X^2] - (E[X])^2
    • Var(c) = 0
    • Var(X + c) = Var(X)
    • Var(cX) = c^2Var(X)
    • Var(X+Y) = Var(X)Var(Y) if X and Y are independent
  3. Covariance
    • Cov(X,Y) = E[(X - E[X])(Y - E[Y])] = E[XY] - E[X][Y]
    • Var(X) = Cov(X,X)
    • Cov(X,c) = 0
    • Cov(aX, bY) = abCov(X,Y)
    • Cov(\vec{X}) = E[(\vec{X} - E[\vec{X}])(\vec{X} - E[\vec{X}])^T] = E[\vec{X}\vec{X}^T] - E[\vec{X}](E[\vec{X}])^T
    • Cov(\vec{X}|\vec{Y}) = E[(\vec{X} - E[\vec{X}|\vec{Y}])(\vec{X} - E[\vec{X}|\vec{Y}])^T|\vec{Y}]
  4. Mean Square Error: MSE(\hat{\theta}) = E[(\hat{\theta} - \theta)^2] = E[(\hat{\theta}-E[\hat{\theta}])^2] + (E[\hat{\theta}]-\theta)^2 = Var(\hat{\theta}) + Bias(\hat{\theta})^2, more detail explanation can be found on wikipedia.
  5. Law of Total Expectation E[X] = E[E[X|Y]]
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s