Ridge Regression

I was struggling with Ridge Regression for a long time because I don’t understand the purpose of it. Until one day I wake up in the morning, I finally realize that Ridge Regression is Normal equation with regularization.

The equation likes :
W = (X^{T}X + \lambda I)^{-1}(X^{T}Y)

Moreover, lambda in this equation does the same function in the last part of the post of Multiple variables linear regression

With bigger lambda, the model becomes more general.
With small lambda, the model may overfitting.

def RidgeRegression(X,mtip,lam=0.001):
	#according to equation
	W = (X.T*X+eye(shape(X)[1])*lam ).I*(X.T*mtip.T)
	return W

You can replace this function with linear regression and see how it goes. Also, Ridge Regression avoid not invertible problems in linear regression. On the other hands, because of regularization, some variables will be eliminated to avoid overfitting. Ridge Regression also be think as shrinkage methods like PCA, which may explain later.

Advertisements

One thought on “Ridge Regression

  1. Pingback: Babyfur Comic: Raichu age regression - My ABDL Life

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s