# Extracting Eigenvalues from A Linear Regression In R

The function below extracts the eigenvalues and an eigenvector from a linear regression model. I wrote this function so that the eigenvalues and vectors are the same as those reported by the SAS PROC REG procedure when using the COLLIN option.

scaleEigen <- function(fit){
# Calculate the eigenvalues and eigenvectors
#
# args: fit - an object of the lm or glm class from which you would
#             like to extract the X matrix
#
# Output: eigenvalues and eigenvectors of the X'X matrix
#
# Method:
#         fit <- lm(y ~ x1 + x2 + x3, data = dataset)
#         summary(fit)
#         scaledEigen(fit)
#
model.data <- fit$model # The data used in the regression Xvec <- as.matrix(cbind(1, model.data[,-1])) # X matrix including intercept Xvec <- apply(Xvec, 2 , function(x) x/sqrt(sum(x^2))) # Rescale the X matrix eigenValues <- eigen(t(Xvec)%*%Xvec) return(eigenValues) }  For example, using cars data from the R dataset package we can extract eigenvalues and eigenvectors of a X'X matrix which can be used to assess multicollinearity of regressors in a linear regression. data(cars) cars.lm <- lm(dist ~ speed, data = cars) summary(cars.lm) # Extract the eigenvalues and eigenvectors from cars.lm using # the function scaleEign above scaleEigen(cars.lm)  This produces the eigenvalues and their associated eigenvectors of a class list as shown in the R output below. $values
[1]    1.94680083          0.05319917

$vectors [,1] [,2] [1,] -0.7071068 0.7071068 [2,] -0.7071068 -0.7071068  You can also assign the output to a an object and access the values using the dollar sign as below: eigenV <- scaleEigen(cars.lm) eigenV$values  # Eigenvalues
eigenV\$vectors  # Eigenvectors


In a regression analysis, if there is one or more small eigenvalues of the X'X, matrix it implies there are near-linear relationships (dependencies) among regressors and we may have multicollinearity issues in the analysis.

Updated: